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Report  To  The  Congress 

OF  THE  UNITED  STATES 


DOD  Needs  To  Provide  More  Credible 
Weapon  Systems  Cost  Estimates 
To  The  Congress 


DOD's  cost  estimates  for  weapon  systems 
programs  are  of  major  concern  when  the 
Congress  is  deciding  to  allocate  billions  of 
dollars  to  defense  programs.  GAO  found 
DOD  cost  estimating  guidance  needs  im¬ 
provement  and  stricter  implementation  to 
ensure  that  cost  estimates  are  uniform, 
consistently  developed,  and  well  docu¬ 
mented.  GAO  also  found  that  using  more 
reasonable  assumptions  and  independent 
cost  estimates  would  result  in  more  ac¬ 
curate  reporting  to  the  Congress. 

This  report  contains  numerous  recommen¬ 
dations  for  improving  DOD  cost  estimating 
and  reporting.  In  addition,  GAO  believes 
that  the  Congress  may  want  to  consider 
requiring  DOD  to  certify  that  estimates  on 
major  systems  are  based  on  sound  cost 
estimating  guidelines  and  represent  the  full 
cost  of  the  weapon  systems  program. 
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The  first  five  copies  of  individual  reports  are 
free  jf  charge.  Additional  copies  of  bound 
audit  reports  are  $3.25  each.  Additional 
copies  of  unbound  report  (i.e.,  letter  reports) 
and  most  other  publications  are  $1.00  each. 
There  will  be  a  25%  discount  on  all  orders  for 
100  or  more  copies  mailed  to  a  single  address. 
Sales  orders  must  be  prepaid  on  a  cash,  check, 
or  money  order  basis.  Check  should  be  made 
out  to  the  "Superintendent  of  Documents". 
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To  the  President  of  the  Senate  and  the 
Speaker  of  the  House  of  Representatives 

This  report  describes  the  Department  of  Defense's  process 
for  developing  and  reporting  weapon  systems  cost  estimates  to 
the  Congress.  It  also  discusses  our  views  and  recommendations 
regarding  how  the  accuracy,  completeness,  and  timeliness  of 
these  estimates  can  be  improved.  We  made  this  review  at  the 
request  of  the  Chairman,  Senate  Committee  on  Governmental 
Affairs . 

We  are  sending  copies  of  this  report  to  interested 
congressional  committees;  the  Director,  Office  of  Management  and 
Budget;  and  to  the  Secretary  of  Defense. 


Comptroller  General 
of  the  United  States 


COMPTROLLER  GENERAL'S 
REPORT  TO  THE  CONGRESS 


DOD  NEEDS  TO  PROVIDE  MORE 
CREDIBLE  WEAPON  SYSTEMS  COST 
ESTIMATES  TO  THE  CONGRESS 


DIGEST 

The  Department  of  Defense's  (DOD's)  cost 
estimates  for  weapon  systems  programs  are  of 
major  concern  when  the  Congress  is  deciding  to 
allocate  billions  of  dollars  to  defense  pro¬ 
grams.  The  accuracy,  completeness,  and  time¬ 
liness  of  DOD's  cost  estimates  need  to  be 
improved  to  give  the  Congress  more  reliable 
data  for  its  decision  process. 

The  Congress  has  expressed  considerable  con¬ 
cern  over  the  years  about  the  effectiveness  of 
DOD's  cost  estimating  processes.  In  response 
to  the  request  by  the  Chairman  of  the  Senate 
Committee  on  Governmental  Affairs,  GAO 
reviewed  DOD's  cost  estimating  and  reporting 
procedures . 

GAO  reviewed  the  DOD  cost  estimating  process 
on  seven  selected  weapon  systems.  GAO's 
sample  includes  three  Army  systems  and  two 
each  from  the  Navy  and  the  Air  Force.  While 
GAO's  sample  does  not  permit  projection  of  the 
results  servicewide  or  DOD-wide,  GAO  believes 
these  cases  illustrate  the  types  of  problems 
that  have  hampered  effective  cost  estimating 
for  weapon  systems,  and  the  lessons  learned 
from  these  systems  can  serve  as  guidelines  for 
improving  the  cost  estimating  process. 

In  the  past  2  years,  DOD  and  the  services  have 
initiated  efforts  to  improve  cost  estimating 
and  reporting.  DOD's  Acquisition  Improvement 
Program  includes  several  initiatives  which,  if 
implemented,  should  improve  cost  estimating. 
These  initiatives  address  program  budgeting, 
inflation  indexes,  and  program  restructuring. 
The  services,  especially  the  Army,  have  also 
begun  to  update  guidance  and  improve  esti¬ 
mating  procedures.  All  these  efforts  are 
needed  to  help  ensure  the  most  effective  esti¬ 
mating  possible.  (See  pp.  46  to  49.) 

COST  ESTIMATES  COULD  IMPROVE  WITH 
BETTER  GUIDANCE,  ASSUMPTIONS, 

AND  METHODOLOGIES 


GAO  found  that  DOD  could  improve  its  cost 
estimating  on  the  selected  systems  by 
(1)  strengthening  and  implementing  its 
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guidance  for  estimating,  (2)  introducing  more 
realism  into  the  assumptions  and  methodologies 
used,  and  (3)  more  fully  using  the  recommenda¬ 
tions  of  DOD ' s  independent  estimating  groups. 

GUIDANCE 


Some  guidance  provided  in  DOD  and  service 
instructions  is  vague  and  conflicting.  DOD 
and  Air  Force  instructions,  for  example, 
allowed  differing  interpretations  of  what 
costs  to  include  in  the  B-1B  program  estimate. 
As  a  result,  some  estimates  included  the  cost 
of  flight  simulators  and  some  did  not.  (See 
pp.  8  and  9 . ) 

Guidance  must  be  more  strictly  implemented  to 
ensure  estimates  are  properly  structured  and 
documented  and  consistently  developed.  GAO 
found  that  in  some  cases  not  all  required 
costs  were  included  in  the  estimates  and  all 
estimates  were  not  updated  as  required. 

On  the  Army's  Apache  helicopter,  for  example, 
$350  million  was  excluded  from  the  production 
decision  program  cost  estimate  for  items  that 
should  have  been  included  under  applicable 
guidance.  Two  Navy  programs  did  not  properly 
document  estimates  as  required  by  Navy  regula¬ 
tions.  (See  pp.  9  to  13.) 

ASSUMPTIONS  AND  METHODOLOGIES 


Assumptions  and  methodologies  used  to  prepare 
estimates  can  greatly  affect  the  cost  out¬ 
come.  Assumptions  regarding  program  risk  are 
one  aspect  GAO  noted  in  its  assessment  of  the 
selected  weapon  systems.  By  assuming  low 
risk,  costs  can  be  held  to  a  minimum.  On  the 
Hellfire  missile  program,  technical  engi¬ 
neering  risks  of  production  were  considered 
low  and  estimates  were  prepared  accordingly, 
and  not  adjusted  for  higher  risk.  GAO's 
January  1983  report  on  the  program,  however, 
pointed  out  higher  risk  associated  with 
several  significant  technical  shortcomings  in 
the  missile  which  could  affect  production 
schedules  and  costs.  (See  pp.  14  and  15.) 

The  treatment  of  inflation  in  estimates  is 
another  area  where  the  assumptions  used  are 
important.  DOD  regulations  require  use  of  the 
Office  of  Management  and  Budget's  (OMB's) 
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developed  inflation  rates,  but  some  estimates 
used  other  rates.  (See  pp.  15  and  16.) 

A  frequently  reported  estimating  problem,  and 
one  which  was  also  illustrated  by  GAO's  case 
studies,  is  the  use  of  overly  optimistic 
assumptions  relating  to  such  things  as 
construction  schedules,  allowances  for 
uncertainties,  fitting  the  estimates  to  the 
services'  budget  contraints,  and  excluding 
certain  relevant  program  costs. 

While  DOD  instructions  call  for  the  use  of 
realistic  factors  and  assumptions,  manage¬ 
ment's  desire  to  keep  cost  estimates  as  low  as 
possible  sometimes  led  to  optimistic  esti¬ 
mates.  For  example,  the  estimate  for  the 
Apache  helicopter  was  reduced  by  $72.6  million 
in  March  1982  to  conform  to  fiscal  year  1983 
budget  guidance.  Estimates  are  kept  low  by 
using  the  most  optimistic  assumptions  regard¬ 
ing  such  factors  as  technical  risk,  develop¬ 
ment  and  production  problems,  and  schedule  and 
design  disruptions.  (See  pp.  17  to  22.) 

INDEPENDENT  ESTIMATES 


The  Office  of  the  Secretary  of  Defense  and  the 
services  have  established  independent  cost 
estimating  groups  to  help  ensure  that  cost 
estimates  are  more  reliable  and  valid.  GAO 
found  some  cases  in  its  sample  in  which 
estimates  by  the  independent  estimators  were 
more  accurate  than  the  official  DOD  estimates, 
but  the  independent  estimators'  recommendations 
were  not  used.  In  one  case,  an  independent 
estimate  pointed  out  that  the  fire  control 
costs  for  the  Apache  helicopter  could  be 
significantly  understated.  This  was  not 
incorporated  into  the  program  estimate.  The 
next  year  the  contractor's  estimate  for  the 
fire  control  system  rose  by  about  $340  mil¬ 
lion — nearly  a  60-percent  increase.  The  1984 
DOD  Authorization  Act  now  requires  the 
Secretary  of  Defense  to  consider  independent 
estimates  and  report  to  the  Congress  regarding 
the  use  of  these  estimates  in  making  decisions 
on  major  acquisition  programs.  (See  pp.  22  to 
24.  ) 

While  the  services  should  not  arbitrarily 
accept  the  recommendations  of  the  independent 
estimators,  GAO  believes  that  their  recommen¬ 
dations  should  be  thoroughly  considered  and  if 
they  are  not  used,  the  services  should 
document  the  reasons  for  not  accepting  the 
independent  estimates. 
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POD  REPORTS  PROVIDED  TO  THE 

CONGRESS  NEED  MORE  REALISM 


DOD  reports  its  estimates  to  the  Congress 
through  reporting  mechanisms  such  as  the 
Selected  Acquisition  Reports  (SARs),  which  are 
quarterly  status  reports  of  the  cost,  sche¬ 
dule,  and  performance  of  the  major  weapon 
systems,  and  the  unit  cost  exception  reports, 
which  show  when  a  system  is  in  danger  of 
exceeding  unit  cost  thresholds  established  for 
that  system.  The  Congress  has  expressed  its 
dissatisfaction  with  the  lack  of  timely  and 
complete  data  in  DOD ' s  reports.  The  Special 
Panel  on  Defense  Procurement  Procedures  noted 
in  its  February  12,  1982,  report  that: 

"the  SAR  is  inadequate  in  its  report¬ 
ing  on  major  weapon  systems  to  the 
Congress,  thus  inhibiting  proper 
oversight.  The  SAR  system  does  not 
consistently  provide  timely  and  com¬ 
plete  information.  The  present  SAR 
system  provides  quarterly  updates 
that  do  not  always  reflect  substan¬ 
tive  fact-of-life  changes  that  may 
have  occurred  in  the  program  since 
the  December  31  SARs."  (See  pp.  84 
and  85 . ) 

GAO  found  that  while  these  reports  provide 
much  useful  information  to  the  Congress,  DOD 
can  improve  them  by  expeditiously  recognizing 
total  anticipated  program  acquisition  costs 
and  reflecting  them  in  the  reports.  For 
example,  the  TOW  missile  system  has  been  a 
part  of  the  Bradley  Fighting  Vehicle  System 
requirements  since  June  1979.  Costs  for  this 
missile  system  were  known  by  the  Bradley 
program  office  in  1982,  but  were  not  included 
in  the  SAR  to  the  Congress  until  December  1983 
when  they  were  officially  approved.  (See 
pp.  38  and  39 . ) 

The  SARs  also  need  to  show  the  total  planned 
acquisition  objectives  being  considered  for 
the  weapon  systems.  In  one  case,  total 
anticipated  quantities  did  not  appear 
sound;  in  others  the  quantities  changed 
annually.  For  the  Navy's  LSD-41  ship  program, 
for  example,  reports  to  the  Congress  indicated 


IV 


a  buy  of  10  ships  while  internal  Navy 
documents  indicated  a  planned  buy  of  16 
ships.  In  the  case  of  Hellfire  missiles,  DOD 
has  been  purchasing  about  6,000  each  year  and 
raised  the  total  planned  quantity  to  be 
purchased,  reported  in  the  SARs,  by  6,000  each 
year.  (See  p.  39 . ) 

GAO  believes  it  is  important  that  DOD ' s 
reports  to  the  Congress  on  weapon  systems 
include  the  most  current  and  realistic 
estimates  of  program  cost  and  status.  Cost 
elements  and  quantities  that  are  clearly 
anticipated  to  be  incurred  as  part  of  the 
program  should  be  disclosed  as  early  as 
possible,  even  though  they  may  not  yet  be 
fully  approved  within  DOD.  Such  disclosure, 
perhaps  in  the  form  of  footnotes  or 
attachments  to  the  SARs,  would  give  the 
Congress  a  better  idea  of  the  complete 
program,  and  still  allow  DOD  flexibility  to 
change  details  of  the  program  as  it 
progresses . 

RECOMMENDATIONS 


GAO  recommends  that  the  Secretary  of  Defense 
improve  weapon  system  cost  estimating  and 
reporting  by  (1)  revising  its  cost  estimating 
guidance  and  basic  data  used  for  estimating 
and  ensuring  stricter  implementation  of  the 
guidance,  (2)  making  fuller  use  of  the 
recommendations  of  DOD ' s  independent 
estimating  groups,  and  (3)  introducing  more 
realism  into  weapon  systems  cost  estimate 
reports  that  are  provided  to  the  Congress. 

GAO's  specific  recommendations  appear  at  the 
end  of  chapters  2  and  3.  (See  pp.  26  to  28 
and  42 .  ) 

MATTERS  FOR  CONGRESSIONAL  CONSIDERATION 


GAO  believes  that  better  weapon  systems  cost 
estimates  are  possible  and  that  earlier  esti¬ 
mates  of  the  realistic  cost  expectations  for 
weapon  systems  can  provide  the  Congress  with  a 
better  basis  on  which  to  make  resource 
allocation  decisions  among  competing  programs. 
In  a  recent  case,  the  Congress  and  its 
oversight  committees  required  the  Secretary  of 
Defense  to  certify  the  validity  of  DOD ' s  cost 
estimates.  Such  certification  could  give  the 
Congress  greater  assurance  that  the  estimates 
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it  receives  are  the  most  complete  and 
realistic  projections  available.  The  Congress 
may  want  to  expand  this  concept  and  require 
the  Secretary  to  certify  that  the  cost 
estimates  which  DOD  provides  on  all  major 
systems  are  prepared  according  to  sound  cost 
estimating  guidelines  and  represent  the  full 
cost  of  the  program.  (See  p.  42.) 

DOD  COMMENTS  AND  GAO  RESPONSE 


DOD  reviewed  a  draft  of  this  report  and 
provided  oral  comments.  Overall,  DOD  did  not 
concur  with  GAO's  findings  and  related 
recommendations  on  cost  estimating  and  on  the 
sufficiency  of  reports  to  the  Congress.  DOD 
was  especially  concerned  with  what  appeared  to 
DOD  to  be  a  fundamental  misunderstanding  by 
GAO  of  the  criteria  for  cost  estimating, 
budgeting,  and  cost  reporting.  DOD  stated 
that  GAO  did  not  recognize  DOD ' s  long-standing 
agreements  with  the  Congress  that  allow  DOD  to 
present  certain  costs  in  budget  lines  separate 
from  the  weapon  systems  budget  line.  DOD  also 
stated  that  its  criteria  for  cost  estimating, 
budgeting,  and  cost  reporting  are  different  to 
meet  specific  needs  of  the  congressional  over¬ 
sight  committees. 

In  the  draft,  GAO  proposed  that  DOD  prepare 
new  instructions  for  cost  estimating  to 
provide  overall  structure  and  to  help  ensure 
consistency.  GAO  has  clarified  and  revised 
that  proposal  to  recognize  that  DOD  has 
volumes  of  instructions,  and  what  is  needed 
are  improvements  in  that  guidance,  not  more 
guidance.  The  examples  noted  in  these  case 
studies  indicate  that  DOD  cost  estimates  are 
not  always  complete  and  consistent,  and  GAO 
believes  improved  guidance  and  better 
implementation  of  it  will  result  in  better 
cost  estimates. 

DOD  stated  that  its  reports  to  the  Congress 
are  consistent  with  long-standing  agreements 
with  the  congressional  committees.  While  some 
of  DOD ' s  reporting  procedures  have  been  agreed 
to  by  specific  congressional  committees,  the 
issues  discussed  in  this  report  show  that 
DOD's  reports  are  still  not  fully  meeting  the 
Congress'  needs  for  complete,  accurate  cost 
estimates.  This  view  is  indicated  by  the  many 
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clear  signals  in  congressional  committee 
reports,  hearings,  and  other  sources  that 
show  dissatisfaction  with  the  information  the 
Congress  gets  from  DOD  on  weapon  systems  cost 
estimates.  This  dissatisfaction  has  continued 
despite  legislation  attempting  to  achieve 
better  reporting — attempts  that  include  SAR 
reporting.  Congressional  Data  Sheets,  and  the 
Nunn/McCurdy  amendments,  which  established  the 
unit  cost  exception  reports.  (See  pp.  37  to 
45  and  app.  VI  item  4.) 

DOD  also  stated  that  the  criteria  for  cost 
estimating  are  different  from  those  for 
budgeting  and  reporting.  The  examples  drawn 
from  GAO's  selected  weapon  systems  illustrate 
there  are  significant  differences  between  the 
costs  DOD  estimates  internally  for  its  weapon 
systems,  and  the  costs  DOD  budgets  and  reports 
to  the  Congress.  While  there  are  sometimes 
good  reasons  for  such  differences,  these 
differences  should  be  disclosed  in  DOD ' s 
reports  and  made  clear  to  the  Congress  so  the 
Congress  has  a  better  understanding  of 
potential  system  costs. 
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GLOSSARY 


Cost  Analysis  Improvement 
Group  (CAIG) 


Cost  estimating  relationship 


Decision  Coordinating 
Paper  (DCP) 


Defense  Systems  Acquisition 
Review  Council  (DSARC) 


Engineering  estimate 


Five-Year  Defense  Program  (FYDP) 


The  principal  advisory  body 
to  the  DSARC  on  matters  of 
cost.  Organizationally  lo¬ 
cated  in  the  Office  of  the 
Secretary  of  Defense,  Direc¬ 
tor  of  Program  Analysis  and 
Evaluation . 

A  mathematical  relationship 
that  defines  cost  as  a  func¬ 
tion  of  one  or  more  parame¬ 
ters  such  as  performance, 
operating  characteristics, 
physical  characteristics, 
etc.  For  example,  past  air¬ 
craft  acquisitions  may  show 
that  the  cost  of  electronic 
components  for  aircraft 
tends  to  be  X  dollars  per 
pound . 

The  principal  document  to 
record  essential  system  pro¬ 
gram  information  for  use  in 
support  of  the  Secretary  of 
Defense  decision  making  pro¬ 
cess  at  the  major  acquision 
milestones . 

An  advisory  body  to  the 
Secretary  of  Defense  on  ma¬ 
jor  system  acquisitions. 

The  Council  members  are  the 
service  secretaries  and  the 
OSD  staff  principals. 

Sometimes  referred  to  as 
"grass-roots"  or  "bottoms- 
up"  estimate.  An  industrial 
engineering  based  approach 
relying  on  detailed  simula¬ 
tion  of  all  the  operations 
and  an  exhaustive  list  cf 
all  the  materials 
required . 

The  Five-Year  Defense  Pro¬ 
gram  summarizes  all  approved 
programs  of  the  entire 
Department  of  Defense. 
Resources  or  inputs  for  5 
years  are  combined  with 
military  outputs  or  programs 


Independent  estimate 


Parametric  estimate 


Production  decision 


Program  Objectives  Memorandum 
(POM) 


for  the  same  period.  FYDP 
is  expressed  in  terms  of 
programs,  program  elements, 
and  resource  categories. 

An  estimate  of  program  cost 
developed  outside  normal 
advocacy  channels  by  a  team 
which  generally  includes 
representation  from  the 
functional  areas  of  cost 
analysis,  procurement, 
production  management, 
engineering  and  program 
management . 

A  cost  analysis  which 
involves  the  development  and 
use  of  estimating  relation¬ 
ships  between  historical 
costs  and  other  program 
variables  such  as  system 
physical /performance 
characteristics,  contractor 
output  measures,  manpower 
loading,  and  facility  floor 
space . 

The  last  of  the  major  mile¬ 
stone  decision  points.  At 
this  point  the  Secretary  of 
Defense  decides  whether  a 
weapon  system  is  to  proceed 
into  final  production  and  be 
deployed  in  the  field. 

A  memorandum  in  prescribed 
format  submitted  to  the 
Secretary  of  Defense  by  the 
Secretary  of  a  Military 
Department  or  the  Director 
of  a  Defense  Agency  which 
recommends  the  total 
resource  requirements  within 
the  parameters  of  the  Secre¬ 
tary  of  Defense  fiscal  gui¬ 
dance  . 

The  standard,  comprehensive, 
summary  status  report  on 
major  defense  acquisition 
programs  which  reflects  the 


Selected  Acquisition  Report  (SAR) 


system  program  manager's 
current  "best  estimate"  of 
key  performance,  schedule, 
and  cost  goals,  compares 
these  estimates  with 
baseline  parameters 
(established  at  the  time  the 
program  was  approved  for 
full-scale  development),  and 
explains  all  variances  from 
the  baseline. 

Unit  Cost  Report  (UCR)  A  report  on  the  status  of 

(1)  program  acquisition  unit 
cost  (total  program  cost 
divided  by  the  total  quan¬ 
tity)  ,  (2)  current  procure¬ 
ment  unit  cost  (current  fis¬ 
cal  year  procurement  cost 
divided  by  quantity  to  be 
procured  in  the  current  fis¬ 
cal  year),  (3)  contract 
costs,  (4)  schedule,  and  (5) 
performance.  The  OCRs  are 
also  to  include  any  known, 
expected,  or  anticipated 
changes  from  the  schedule 
milestones  or  operational 
and  technical  characteris¬ 
tics  shown  in  the  baseline 
program.  It  is  submitted  by 
the  program  manager  on  each 
SAR  program  at  the  end  of 
each  calendar  quarter  or 
when  the  program  manager 
expects  the  unit  costs  to 
breach  the  thresholds  des¬ 
cribed  in  the  UCR  legisla¬ 
tion  . 


CHAPTER  1 


INTRODUCTION 


Cost  estimates  for  major  weapon  systems  are  a  crucial  part 
of  the  acquisition  process,  especially  in  a  resource-scarce 
environment.  If  decisionmakers  are  to  exercise  sound  judgment 
about  the  affordability  of  weapon  systems,  high  quality  esti¬ 
mates  of  weapon  systems'  costs  must  be  generated  by  the  Depart¬ 
ment  of  Defense  (DOD)  and  the  services.  For  the  estimates  to  be 
credible  and  useful  to  decisionmakers,  they  must  be  developed 
under  clearly  consistent  and  precise  guidance  which  specifies 
the  estimates'  purpose,  basis,  and  methodology. 

CONGRESSIONAL  REQUEST 

In  May  1982,  the  Chairman,  Senate  Committee  on  Governmental 
Affairs,  requested  us  to  assess  the  effectiveness  of  DOD  cost 
estimating  processes  for  major  weapon  systems.  The  Chairman 
expressed  concern  over  the  cost  estimating  process,  the  use  of 
contractor  data,  and  the  development  of  DOD's  official  cost 
estimates.  The  Chairman  asked  us  to  examine  several  actual 
examples  of  how  the  cost  estimating  process  worked  for  specific 
major  acquisitions.  (See  app.  I.) 

We  selected  seven  weapon  systems  as  case  studies — two  Air 
Force,  two  Navy,  and  three  Army  acquisition  programs.  We  selec¬ 
ted  some  weapon  systems  in  early  development  and  some  in  produc¬ 
tion.  To  meet  the  Chairman's  request,  we  reviewed  the  estimat¬ 
ing  process  from  the  estimates  of  the  program  office  cost  esti¬ 
mators,  through  the  DOD  hierarchy,  to  the  reporting  of  those 
estimates  to  the  Congress. 

THE  PURPOSE  OF  DOD  COST  ESTIMATES 


Various  regulations  and  directives  of  the  Office  of  the 
Secretary  of  Defense  (OSD)  and  the  services  govern  the  cost 
estimating  process.  They  are  intended  to  specify  which  organi¬ 
zational  element  develops  estimates  for  what  purpose,  how  the 
estimates  are  to  be  reviewed  and  validated,  and  how  they  are  to 
be  reported  to  the  Congress.  (See  app.  II.) 

The  directives  emphasize  affordability,  priority,  and 
availability  of  fiscal  and  manpower  resources.  At  each  acquisi¬ 
tion  milestone,  the  investment  the  service  is  willing  to  commit 
and  the  program  priority  is  reconciled  with  overall  capabili¬ 
ties,  priorities,  and  resources.  Directives  specify  that  system 
acquisition  planning  shall  be  based  on  adequate  program 
funding.  Programs  are  not  to  proceed  into  concept  exploration 
or  demonstration  and  validation  unless  money  is  available. 
Approval  to  proceed  into  full-scale  development  or  production 
also  depends  on  the  availability  of  money  at  those  times. 
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To  ensure  a  maximum  cost/benefit  in  systems  acquisition, 
each  service  strives  for  the  same  ultimate  objective — to  have 
high  quality  cost  estimates  available  to  decisionmakers.  In 
line  with  this  objective,  service  regulations  have  specified  the 
role  and  function  of  cost  estimating. 

DOD  Directive  5000.4  assigns  to  the  Cost  Analysis  Improve¬ 
ment  Group  under  the  Director,  Program  Analysis  and  Evaluation, 
the  primary  responsibility  in  OSD  for  cost  estimating.  This 
group  provides  the  Defense  Systems  Acquisition  Review  Council 
(DSARC)  with  an  evaluation  of  service  independent  and  program 
office  cost  estimates.  (See  app.  III.) 

CRITERIA  BASIC  TO  AN 
EFFECTIVE  ESTIMATING  PROCESS 

Our  last  comprehensive  review  of  DOD  cost  estimating 
policies  and  practices  resulted  in  a  July  24,  1972,  report  to 
the  Congress  entitled  Theory  and  Practice  of  Cost  Estimating  for 
Major  Acquisitions  (B-163058).  The  report's  principal  findings 
and  conclusions  were  that  uniform  DOD  guidance  on  cost  estima¬ 
ting  was  lacking.  We  also  found  documents  to  be  inaccurate,  and 
readily  retrievable  cost  data  for  computing  estimates  was  lack¬ 
ing.  We  suggested  nine  criteria  for  effective  estimating: 

— Clearly  identify  the  task. 

— Ensure  broad  participation  in  preparing  estimates. 

— Ensure  valid  data  is  available. 

— Standardize  the  structure  for  estimates. 

— Provide  for  program  uncertainties  (risks). 

— Properly  recognize  inflation. 

— Recognize  excluded  costs. 

— Independently  review  estimates. 

— Revise  estimates  when  significant  program  changes  occur. 
These  criteria  are  explained  in  appendix  IV. 

OSD  and  the  services  have  taken  steps  to  improve  their  cost 
estimating.  OSD's  efforts  are  part  of  the  DOD  Acquisition 
Improvement  Program.  The  services,  particularly  the  Army,  also 
have  efforts  underway  to  improve  their  cost  estimating  as  dis¬ 
cussed  in  detail  in  chapter  4.  (See  pp.  46  to  49.) 

CRITICISM  OF  DOD  COST  ESTIMATING 

DOD  has  been  widely  criticized  for  the  increasing  cost 
growth  in  major  weapon  systems  acquisition.  For  instance,  the 
American  Defense  Preparedness  Association  report  of  The  Results 
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of  a  Cost  Discipline  Conference,  held  July  21  to  22,  1982, 
concluded : 

"Cost  estimates  on  weapons  systems  development  pro¬ 
grams  are  not  well  done,  are  not  updated  with  suffici¬ 
ent  frequency,  and  are  characterized  by  being  overly 
optimistic,  particularly  at  the  early  stages  of  the 
program.  Many  of  these  problems  stem  from  the  compet¬ 
itive  nature  of  the  DOD  budget,  defense  industry  com¬ 
petition,  and  the  contracting  process." 

In  another  view,  the  U.S.  Army  Audit  Agency  report  HQ 
82-709,  of  April  30,  1982,  on  four  missile  command  programs 
states ; 

"Program  cost  estimates  for  the  Hellfire,  Pershing  II, 
and  Viper  understated  future  costs  of  producing  the 
systems. 

"The  cost  estimating  process  for  development  and  pro¬ 
duction  contracts  was  not  adequate. 

"Overall  guidance  on  preparation  of  independent  gov¬ 
ernment  cost  estimates  is  lacking. 

"For  the  Hellfire,  Multiple  Launch  Rocket  System,  and 
Pershing  II,  breaches  to  cost  thresholds  were  either 
not  reported  or  not  reported  promptly  in  revised 
decision  coordinating  papers  to  Army/Defense  Systems 
Acquisition  Review  Councils  for  their  required  review 
and  approval.  .  .  .  Long  delays  were  also  experienced 

in  reporting  large  cost  increases  in  the  Selected 
Acquisition  Reports." 

The  February  12,  1982,  report  of  the  Special  Panel  on 
Defense  Procurement  Procedures  of  the  House  Armed  Services  Com¬ 
mittee  (McCurdy  Panel)  found: 

"The  factors  contributing  to  cost  growth  are  unrealis¬ 
tic  inflation  estimates,  poor  cost  estimates,  program 
stretch-outs,  changes  in  specifications,  inadequate 
budgeting,  high  risk  system  design,  poor  management, 
and  lack  of  competition  ....  (Emphasis  Supplied.) 

"The  present  SAR  [Selected  Acquisition  Report]  system 
provides  quarterly  updates  that  do  not  always  reflect 
substantive  fact-of-life  changes  that  may  have 
occurred  in  a  program  since  the  December  31  SARs." 

OBJECTIVES,  SCOPE,  AND  METHODOLOGY 


Cost  growth  on  major  weapon  systems  has  been  recognized  as 
a  long-standing  problem  by  all  the  players  involved,  including 
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DOD,  the  Congress,  military  contractors,  the  Congressional 
Budget  Office,  our  office,  and  so  forth.  The  estimating  process 
is  frequently  cited  as  a  significant  contributor  to  understated 
costs. 


We  took  a  broad  approach  to  this  review  of  the  estimating 
process.  Our  objectives  were  to  look  for  instances  where  occur¬ 
rences  or  factors  causing  cost  growth  should  have  been 
anticipated  by  the  program  office,  the  services,  or  OSD,  but 
were  not,  or  where  such  items  were  purposely  not  considered,  or 
where  actions  were  directed  to  create  the  appearance  of  reduced 
program  costs.  We  did  not  limit  our  review  to  instances  of  cost 
growth  directly  attributable  to  cost  estimating  methodologies. 
Our  review  included  all  cost  growth  contributors  to  determine 
why  they  were  not  recognized  earlier  in  the  life  of  the  pro¬ 
gram.  Our  approach  was  to  examine  DOD's  cost  estimating  process 
to  determine  what  improvements  can  be  made  regarding  estimating 
techniques  and  processes,  management  of  the  process,  and  the 
environment  in  which  the  estimates  are  made. 

Our  conclusions  and  recommendations  are  based  on  the  prob¬ 
lems  observed  in  DOD's  cost  estimating  process  on  seven  selec¬ 
ted  weapon  systems.  We  selected  these  systems  beginning  with 
the  B-1B  bomber  because  our  previous  work  on  the  B-1B  brought 
the  issue  of  DOD  cost  estimating  to  the  attention  of  the 
Chairman,  Senate  Committee  on  Governmental  Affairs.  We  decided 
that  the  other  systems  in  our  sample  would  be  selected  to 
include : 

— systems  from  all  three  services; 

— different  types  of  weapon  systems,  that  is  missiles,  air¬ 
craft,  tracked  vehicles,  and  ships; 

— systems  in  different  phases  of  the  acquisition  process; 

— systems  which  were  major  and  nonmajor — as  defined  by  the 
SAR  reporting  requirements  in  effect  at  the  time  of  our 
review;  and 

— systems  that  were  not  the  subject  of  ongoing  reviews  by 
us  or  the  audit  agencies  of  OSD  or  the  services. 
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We  selected  the  following  systems; 


System  name 

Service 

Weapon 

type 

Phase  of 
acquisition 

Apache  Helicopter 

Army 

Aircraft 

Production 

Bradley  Fighting 

Vehicle  System 

Army 

Tracked  vehicle 

Production 

Hellfire  Missile 

Army 

Missile 

Production 

Light  Airborne  Multi- 
Purpose  System  (LAMPS) 
MK  III  Helicopter 

Navy 

Aircraft 

Production 

Landing  ship  dock 
(LSD) -41 

Navy 

Ship 

First  three  ships 
under  contract 

T-46A  Trainer 

Air  Force 

Aircraft 

Development 

B-1B  Bomber 

Air  Force 

Aircraft 

Development 

Descriptions  of  these  systems  appear  in  appendix  V. 

While  our  conclusions,  and  hence  our  recommendations, 
result  from  our  review  of  a  relatively  small,  nonscientif ic  sam¬ 
ple  of  weapon  systems  which  does  not  permit  projection  of  the 
results  servicewide  or  DOD-wide,  we  believe  that  the  problems  we 
observed  on  those  systems  are  consistent  with  the  findings  of 
other  studies  and  illustrate  the  types  of  problems  that  have 
hampered  effective  cost  estimating  for  weapon  systems.  Appendix 
VI  is  a  listing  and  brief  synopsis  of  pertinent  studies, 
reports,  and  other  writings  we  reviewed  during  our  study  that 
address  similar  cost  estimating  problems.  Appendix  VII  lists 
our  past  reports  that  address  the  area.  Our  examples — drawn 
from  the  in-depth  review  of  the  cost  estimating  process  for 
seven^  systems — serve  as  illustrations  of  problems  that  have 
been  discussed  in  these  studies.  While  those  broad-based  stud¬ 
ies  indicate  a  problem  exists,  our  in-depth  analysis  provides 
concrete  examples  of  the  problems  and  the  lessons  learned  from 
these  examples  that  can  serve  as  guidelines  for  improving  the 
cost  estimating  process.  These  examples  show  problems  in  all 
three  services,  and  we  have  not  attempted  to  compare  the 
services  or  evaluate  which  is  doing  a  better  estimating  job. 


^The  T-46A  is  in  the  early  stages  of  the  acquisition  process. 
Therefore,  a  cost  history  was  not  available  on  the  T-46A,  and 
for  this  reason,  it  was  not  used  as  an  example  of  cost 
estimating  problems  in  this  report. 
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In  doing  our  review,  we  obtained  and  reviewed  applicable 
DOD ,  service,  and  major  command  regulations,  directives, 
instructions,  guides,  pamphlets,  circulars,  policy  and  procedure 
statements,  memorandums,  and  correspondence  to  describe  the  cost 
estimating  process.  We  also  interviewed  cognizant  OSD,  service 
headquarters  and  commands,  and  program  office  representatives. 

To  trace  the  most  recent  major  costing  effort  and  identify 
all  the  major  costing  efforts,  we  interviewed  representatives  of 
OSD,  the  service  headquarters  and  commands,  the  program  office, 
contractors,  the  service  plant  representative  offices,  and  the 
Defense  Contract  Audit  Agency  at  contractor  locations.  We  also 
obtained,  reviewed,  and  compared  applicable  cost  estimates, 
validations,  analyses,  briefing  charts,  studies  and  reports, 
decision  and  position  papers  and  memorandums,  system  specifica¬ 
tions,  minutes  of  meetings,  correspondence,  and  supporting  docu¬ 
ments. 

We  obtained,  reviewed,  and  compared  applicable  SARs,  unit 
cost  reports,  and  applicable  unit  cost  exception  reports.  We 
reviewed  applicable  guidance  and  compared  the  reports  to  current 
cost  estimates.  Also,  we  interviewed  representatives  of  OSD, 
service  headquarters  and  commands,  and  the  program  offices  to 
evaluate  the  reports. 

To  assess  the  status  of  recent  service  cost  estimating 
initiatives,  we  obtained  and  reviewed  service  audit  agency 
reports,  special  study  reports,  a  service  Memorandum  of  Under¬ 
standing,  recent  command  circulars,  and  service  headquarters  and 
command  letters  of  instruction.  We  also  interviewed  service 
headquarters  and  commands,  and  program  office  representatives. 

The  audit  work  for  this  review  required  visits  to  contrac¬ 
tor  locations,  program  offices,  testing  facilities,  command  and 
service  headquarters,  DOD  Headquarters,  and  OSD.  (See 
app.  VIII.) 

DOD  comments  are  discussed  at  the  end  of  the  relevant  chap¬ 
ters.  We  also  asked  the  American  Defense  Preparedness 
Association  to  review  our  draft  report.  Their  written  comments 
are  presented  in  appendix  IX.  While  the  American  Defense 
Preparedness  Association  questioned  whether  our  examples  were 
too  old  to  demonstrate  the  current  situation,  it  still  agreed 
with  the  overall  message  of  our  report. 

Appendix  X  includes  analyses  of  the  December  1982  SAR 
reports  for  the  36  programs  for  which  the  impact  of  the  produc¬ 
tion  decision  could  be  identified  in  past  SARS.  Four  of  our 
selected  systems — the  Hellfire  missile,  the  Apache  Helicopter, 
the  Bradley  Fighting  Vehicle  System,  and  the  B-1B  Bomber — are 
included  in  these  analyses.  An  explanation  of  our  methodology 
for  these  analyses  is  included  in  the  appendix. 
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Our  review  was  performed  in  accordance  with  generally 
accepted  government  auditing  standards.  The  review  was  initi¬ 
ated  in  July  1982,  and  fieldwork  completed  by  March  31,  1983. 
The  information  in  this  report  has  been  updated  to  take  into 
account  changes  since  the  completion  of  fieldwork. 
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CHAPTER  2 


POD  NEEDS  BETTER  GUIDANCE, 

ASSUMPTIONS,  AND  METHODOLOGIES  FOR  ESTIMATING 

Logically,  cost  estimators  should  prepare  their  estimates 
by  using  basic  source  data  and  working  within  the  structure  pro¬ 
vided  by  OSD  and  service  guidance.  Although  many  of  the  criti¬ 
cisms  of  cost  estimating  are  aimed  at  the  cost  estimators,  some 
of  the  factors  with  the  greatest  effect  on  cost  estimates  are 
beyond  their  control.  The  DOD  hierarchy  tells  them  what  to 
estimate  and  what  techniques  and  basic  data  to  use  in  developing 
their  estimate.  The  examples  provided  by  our  selected  weapon 
systems  show  that: 

In  some  cases,  OSD  and  the  services  need  to  improve  their 
cost  estimating  guidance. 

OSD  and  the  services  need  to  improve  the  implementation 
of  their  cost  estimating  guidance.  On  some  of  the  seven 
systems  we  reviewed,  estimates  were  not  structured  as 
required  under  current  regulations;  others  were  not  docu¬ 
mented  as  required  to  verify  how  the  estimates  were 
developed,  and  to  trace  from  one  estimate  to  the  next. 

Improvements  are  needed  in  the  assumptions  and  methodolo¬ 
gies  used  to  develop  an  estimate. 

Improvements  are  needed  in  the  assumptions  often  recom¬ 
mended  by  DOD  management. 

Independent  cost  estimators'  estimates  and  recommenda¬ 
tions  are  not  always  adequately  used. 

DOD  NEEDS  TO  IMPROVE  ITS 

COST  ESTIMATING  GUIDANCE 


The  services  have  developed  their  own  cost  estimating 
guidance.  In  some  cases,  we  found  the  service  guidance  was 
vague  and  in  other  cases  it  conflicted  with  OSD  guidance. 

Vague  cost  estimating  guidance  is  shown  by  the  Army's  cost¬ 
ing  guides.  For  example,  these  guides,  in  addressing  the  ques¬ 
tion  of  what  costs  should  be  attributed  to  a  specific  system, 
state 


"if  a  given  component  would  not  exist  if  the  system 
did  not  exist,  then  that  component  must  be  included 
in  the  definition  of  the  materiel  system." 

Such  a  broad  generalization  appears  to  us  to  be  subject  to 
varying  interpretations. 


8 


Conflicting  service  guidance  has  a  severe  effect  on  program 
office  estimates,  particularly  when  these  problems  affect  the 
definition  of  the  system  to  be  estimated.  The  Air  Force's  B-1B 
bomber  program  has  suffered  from  some  of  these  problems. 

The  B-IB  program  office  was  told  by  the  Secretary  of 
Defense  to  exclude  certain  items  (e.g.  the  simulator^)  from  the 
B-IB  program  so  that  the  President  could  certify  that  the 
program  costs  would  not  exceed  $20.5  billion.  The  B-IB  program 
office  estimate  excluded  these  items,  but  the  independent  Air 
Force  estimate  and  OSD  Cost  Analysis  Improvement  Group  estimates 
included  them.  The  B-IB  SAR  does  not  include  these  items  or 
their  costs,  but  lists  them  as  related  programs. 

On  July  22,  1982,  we  testified  before  the  Defense  Subcom¬ 
mittee  of  the  House  Appropriations  Committee  on  the  differences 
in  the  B-IB  program  office  estimate,  the  independent  Air  Force 
estimates,  and  the  OSD  Cost  Analysis  Improvement  Group  esti¬ 
mates.  One  of  the  problems  we  noted  was  conflicting  guidance  on 
what  is  to  be  included  in  the  program.  For  example,  DOD 
Instruction  5000.33  indicates  that  some  costs  can  be  excluded 
from  the  program  office  estimate  when  funded  by  a  separate  bud¬ 
get  line.  In  contrast.  Air  Force  Systems  Command  Regulation 
500-18  indicates  that  the  program  office  estimate  is  to  include 
all  directed  effort  for  which  the  program  office  has  management 
responsibility,  regardless  of  the  source  of  funds.  We  believe 
this  guidance  allows  too  much  latitude  to  the  program  manager 
and  it  is  not  definitive  enough  to  ensure  uniformity  of  cost 
estimates  presented  to  the  Congress. 

IMPLEMENTATION  OF  GUIDANCE  IS  WEAK 

DOD  needs  to  improve  the  implementation  of  its  cost 
estimating  guidance.  Our  selected  weapon  systems  provided 
illustrations  of  problems  resulting  from  poor  implementation, 
including 


— a  lack  of  definite  and  consistent  cost  estimate 
structure  and 

—  inadequate  documentation  of  cost  estimates. 

Cost  estimates  lack 

definite  structure 


If  cost  estimates  are  to  be  useful  to  decisionmakers,  they 
must  have  a  definite  and  consistent  structure.  Such  a  structure 
would  ensure  that  all  costs  associated  with  a  weapon  system  are 


^We  were  told  that  the  current  Air  Force  policy  is  to  fund 
simulator  development  under  the  Simulator  System  Project  Office 
program  element,  excluding  this  cost  from  the  program  office 
estimate . 
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included  in  the  estimate,  and  that  the  estimates  consistently 
include  all  associated  costs  from  one  period  to  the  next. 
However,  we  found  that  in  some  cases  what  is  included  in  the 
weapon  system  estimate  is  not  consistent  between  estimates. 

Lack  of  consistent  program  definition 

DOD  and  service  guidance  establish  and  define  the  work 
breakdown  structures  to  be  used  for  cost  estimates  of  major  wea¬ 
pon  system  acquisitions.  A  work  breakdown  structure  is  a 
detailed  description  of  the  tasks  to  be  accomplished  to  achieve 
a  specified  product.  Military  Standard  881-A  establishes  the 
work  breakdown  structure,  uniform  definitions,  and  a  way  to 
develop  the  upper  three  work  breakdown  structure  levels.  The 
Air  Force  Systems  Command  Manual  173-1  and  Army  Pamphlets  11-2 
through  11-5  also  establish  the  basic  pattern  for  estimates  and 
give  work  breakdown  structure  matrices  of  cost  structures  to  be 
used.  Despite  this  guidance,  problems  occur  regarding  what  is 
included  in  the  program  definition,  and  the  program  cost  esti¬ 
mates  are  not  consistent. 

An  example  that  illustrates  these  problems  occurred  on  the 
Army's  Apache  helicopter.  The  production  decision  program  cost 
estimate  for  the  Apache,  although  based  on  Military  Standard 
881-A,  excluded  $291  million  in  research,  development,  and 
investment  costs  associated  with  the  combat  mission  simulator, 
(required  under  881-A  element  40.2.2.1  app.  A)  and  $59  million 
in  costs  associated  with  aircraft  survivability  equipment 
(required  under  881-A  elements  40.2.1.7  and  40.2.1.15  app.  A). 

Another  example  occurred  on  the  Army's  Bradley  Fighting 
Vehicle  System.  Different  structures  were  used  to  develop  the 
1979  and  1982  program  cost  estimates  for  the  Bradley: 


(mill ions ) 
$13,358.7 


1982  program  office  estimate 
1979  program  office  estimate 


7,742.0 

$  5,616.7 


Difference 


The  two  estimates  were  not  prepared  at  the  same  work  break¬ 
down  structure  level  and  were  not  consistent  in  the  cost  ele¬ 
ments  included.  The  1979  program  office  estimate  was  used  to 
support  the  production  milestone  decision,  and  the  1982  program 
office  estimate  was  developed  for  the  Army's  new  Program  Manage¬ 
ment  Control  System.  Both  estimates  are  based  on  contractor 
data,  but  in  1979  the  contractor  used  the  approved  work  break¬ 
down  structure  (that  is,  hull,  suspension,  power  package,  etc.). 
However,  in  the  1982  estimate,  the  contractor's  manufacturing 
costs  were  broken  out  by  the  contractor's  internal  reporting 
structure,  not  by  the  work  breakdown  structure  used  in  the  1979 
estimate.  Thus,  the  two  estimates  are  not  comparable.  Without 
consistently  structured  estimates,  the  Bradley  Fighting  Vehicle 
System  program  office  cannot  explain  why  costs  increased  $5.6 
billion  between  these  two  estimates. 
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Systems  not  defined 


The  services  often  change  their  total  program  cost 
estimates  by  adding  or  deleting  costs.  Program  "restructuring" 
has  become  an  acceptable  reason  for  changing  the  program's  total 
cost.  Although  restructuring  is  done  for  legitimate  reasons  in 
some  cases,  in  others  it  simply  presents  a  lower  total  cost. 

The  following  example  illustrates  how  program  costs  are  added  to 
a  program  estimate  and  then  deleted  from  a  later  estimate. 

The  September  1979  vehicle  investment  estimate  for  the 
Bradley  Fighting  Vehicle  System  excluded  several  product 
improvements  identified  earlier  in  the  June  1979  system 
requirements  on  which  the  estimate  was  based.  These  same 
product  improvements  were  later  added  to  the  program  cost 
estimate  in  the  December  1980  SAR  with  a  projected  cost  increase 
of  $740.9  million  as  shown  below; 


(millions) 
$  9.982 


Low  profile  antenna 
Heading  reference  unit 
Thermal  driver  viewer 
Laser  rangefinder 
Swim  barrier 

Biological/chemical  protection 


49.244 

221.303 

154.458 

30.890 

275.187 


$740. 900^ 


^Does  not  add  due  to  rounding  by  fiscal  year  in  the  SAR. 

These  same  items,  although  they  were  included  in  the  March/July 
1982  program  office  estimate,  were  deleted  from  the  December 
1982  SAR  estimate  because  they  are  no  longer  part  of  the 
approved  program,  and  a  cost  reduction  of  $982.1  million, 
including  inflation,  was  reported. 

Similarly,  the  ammunition  costs  for  the  Bradley  System  were 
reported  inconsistently.  The  1979  program  estimate  showed 
ammunition  costs  of  $423.2  million.  This  figure  was  increased 
by  $542.8  million  in  the  December  1980  SAR  estimate.  In  1982 
DOD  decided  to  report  ammunition  costs  as  a  separate  budget 
item  because  the  25mm  ammunition  was  not  unique  to  the  Bradley. 
Ammunition  costs  were  totally  deleted  from  the  December  1982 
Bradley  SAR,  resulting  in  a  cost  reduction  of  $1,160.2  million. 
No  explanation  was  provided  for  the  mathematical  inconsistencies 
in  the  figures. 

Estimate  documentation  is  insufficient 

DOD  and  service  regulations  require  documented  cost  esti¬ 
mates  to  allow  traceability  from  one  estimate  to  the  next,  allow 
verification,  and  maintain  cost  discipline.  An  earlier  edition 
of  DOD  Instruction  5000.2,  for  example,  stated; 


"Traceability  of  successive  cost  estimates  .  .  .  shall 

be  maintained  starting  with  the  program  cost  estimates 

approved  at  Milestone  I." 

The  current  version  of  DOD  Instruction  5000.2  does  not 
include  this  statement.  DOD  Instruction  5000.2  has  been  short¬ 
ened  from  20  pages  to  7  pages,  and  the  new  version  deletes  many 
of  the  specifics  concerning  cost  estimating.  Similarly,  Air 
Force  Regulation  550-18  states  that  the  program  office  estimate 
has  two  main  objectives; 

"(1)  to  establish  and  maintain  program  cost  discipline 
and 

"(2)  to  provide  an  unequivocal  cost  track." 

Our  1972  report  discussed  problems  in  DOD ' s  documentation 
of  data  sources,  assumptions,  methods,  and  decisions  basic  to 
the  weapon  systems  cost  estimates.  We  found  similar  problems  on 
some  of  our  selected  weapon  systems.  The  following  examples  are 
of  systems  with  documentation  that  could  not  be  used  to  evaluate 
the  estimating  procedures  or  explain  the  reasons  for  cost  growth 
from  one  estimate  to  the  next. 

The  Navy's  LSD-41  and  LAMPS  MK  III 

helicopter  programs 

The  Navy  programs  we  reviewed  did  not  document  their 
estimates  as  required  by  Navy  regulations.  The  principal  Naval 
Sea  Systems  Command,  Naval  Material  Command,  and  Chief  of  Naval 
Operations  and  staff  cost  estimating  regulations  mandate  a  "Cost 
Estimate  Documentation  Summary"  (Naval  Material  Command  Form 
7000/2)  which  summarizes  weapons  system  characteristics, 
procurement  quantity,  and  assumed  production  quantity.  A  Navy 
Decision  Coordinating  Paper  and  a  statement  about  the 
estimator's  confidence  in  the  estimate  are  also  required. 

A  Cost  Estimate  Documentation  Summary  was  not  prepared  on 
the  LSD-41  program.  The  only  documentation  for  the  multi¬ 
billion  dollar  program  estimate  was  the  budget  backup  material 
which  included  little  of  the  required  information.  Cost  and 
program  information  in  the  Navy  Decision  Coordinating  Paper  was 
outdated.  No  statement  regarding  the  confidence  the  Naval  Sea 
Systems  Command  has  in  the  estimate  was  included  except  for  the 
assertion  that  the  program  was  low  risk. 

Navy  regulations  further  state  that,  "documentation  of 
estimates  is  the  key  to  tracking  of  weapon  systems  costs."  The 
Chief  of  Naval  Operations  Instruction  7000. 17A  requires  a  cost 
profile  track  for  each  program  that  traces  the  history  of  costs 
involved  in  the  program,  from  concept  formulation  to  the  present 
time.  This  also  displays  funding  changes  and  their  rationale. 
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No  such  cost  profile  track  was  available  on  the  LSD-41 
program.  Estimate  tracking  has  been  done  through  budget  docu¬ 
ments  since  the  program  entered  production  in  fiscal  year  1981. 
The  LSD-41  program  office  had  difficulty  providing  documents  for 
tracking  program  cost  estimates  because  of  changes  in  cost  esti¬ 
mating  staffs. 

The  Navy  used  an  iterative  approach  to  its  cost  estimates 
on  the  multibillion  dollar  LAMPS  MK  III  helicopter  program. 
Whenever  new  information  was  developed,  the  previous  estimate 
was  updated.  The  Navy  did  this  even  for  the  production  decision 
which  took  place  in  September  1981.  The  tracking  system  for 
these  iterations  is  informal,  consisting  largely  of  internal 
file  folders.  Whether  the  tracking  can  or  cannot  be  reproduced 
depends  on  the  recordkeeping  of  the  individual  estimator.  We 
found  no  formal  documents  to  tie  the  Decision  Coordinating  Paper 
and  SAR  cost  estimates  into  a  work  breakdown  structure  explain¬ 
ing  how  estimates  were  made,  or  referral  to  specific  pricing 
data  inputs. 

The  Army's  Bradley  Fighting  Vehicle  System  program 

Again,  the  purpose  of  documentation  is  to  provide  the  means 
for  tracking  program  cost  estimates  and  enforcing  cost  disci¬ 
pline.  Without  such  documents,  tracking  of  estimates  over  the 
period  of  the  system  development  span  is  impossible,  and  changes 
to  the  estimate  due  to  quantity,  engineering,  and  schedule  can¬ 
not  be  explained. 

For  the  Bradley  Fighting  Vehicle  System,  the  1979  program 
cost  estimate  was  the  cost  projection  used  to  support  the  pro¬ 
duction  milestone  decision.  It  was  not  the  bench  mark  for 
tracking  and  controlling  system  costs.  It  had  no  other  apparent 
purpose  than  to  meet  the  Army  Regulation  11-18  requirement  that 
a  program  cost  estimate  be  prepared  in  support  of  the  Army  Sys¬ 
tems  Acquisition  Review  Council  (ASARC)  and  the  DSARC  for  the 
production  milestone  decision. 

Officials  of  the  Bradley's  program  management  division 
stated  that  they  used  the  budget  figures  reported  in  the  SAR  to 
track  costs  rather  than  the  program  cost  estimate  developed  for 
the  DSARC  process.  For  instance,  they  say  the  budget  for  the 
fiscal  year  1982  approved  program  appearing  in  the  December  1980 
SAR  was,  in  essence,  a  revised  program  cost  estimate.  Officials 
maintain  that  the  SAR  variance  analysis  provides  a  cost  track 
back  to  the  December  1979  SAR  (fiscal  year  1981  approved  pro¬ 
gram),  and  that  this  is  all  that  is  necessary  for  cost  tracking. 

The  result  is  that  the  program  office  can  track  costs  only 
in  general  terms  and  only  from  SAR  to  SAR.  They  cannot  track  or 
explain  in  detail  the  $5.6  billion  increases  from  the  $7.7  bil¬ 
lion  estimate  of  1979  to  the  $13.3  billion  estimate  of  1982. 

( See  p.  10.) 
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ASSUMPTIONS  AND  METHODOLOGIES 
NEED  IMPROVEMENT 


Examples  from  weapon  systems  we  reviewed  illustrate 
weaknesses  in  the  assumptions  and  methodologies  used  to  develop 
cost  estimates  with  regard  to 

— the  recognition  of  program  risk  and 

— consistency  of  inflation  recognition. 

Recognition  of  program  risk 

Estimators  should  identify  the  risks,  determine 
their  probability,  and  increase  the  amount  of  the  estimate  by 
the  magnitude  of  the  risk.  We  found  that  cost  estimators  base 
their  estimates  on  the  information  available  to  them  at  the 
time.  Early  estimates  are  often  optimistic,  or  success  orien¬ 
ted,  and  the  risk  factor  included  in  them  to  provide  for  uncer¬ 
tainties  is  probably  too  low.  Thus,  when  the  early  cost  esti¬ 
mates  are  compared  to  later  estimates,  one  typically  sees  large 
cost  increases.  This  is  illustrated  by  the  SAR  data  displayed 
in  appendix  X.  Cost  estimators  assume  that  the  system  they  are 
estimating  will  not  suffer  from  typical  changes  in  scheduling, 
funding,  engineering,  or  the  threat. 

The  February  9,  1983,  Air  Force  report — the  Affordable 
Acquisition  Approach — comes  to  a  similar  conclusion.  It  shows 
that  most  weapon  systems  will  suffer  from  these  kinds  of 
changes.  The  Affordable  Acquisition  Approach  concludes  that 
early  program  estimates  need  to  include  provisions  for  histori¬ 
cally  proven  growth  such  as  changes,  unknown  development  prob¬ 
lems,  overly  optimistic  baselines,  and  so  forth. 

Army  estimators  told  us  the  most  difficult  cost  estimating 
job  is  to  make  sure  that  estimates  remain  synchronized  with 
changing  requirements.  However,  frequently  changing  require¬ 
ments  make  it  very  difficult  to  keep  them  in  line.  The  Army  is 
trying  to  provide  for  changes  in  programs  through  a  system 
called  Total  Risk  Assessing  Cost  Estimating.  (See  p.  48.) 

Army's  Hellfire  missile  program 

The  Hellfire  production  estimate  provides  an  example  that 
shows  some  estimates  do  not  include  all  the  necessary  provisions 
for  risk.  The  program  and  independent  estimates  do  not  include 
technical  analyses  addressing  engineering  risk.  Army  officials 
told  us  they  did  not  perform  an  engineering  risk  analysis  for 
the  production  phase  because  at  that  time  it  was  not  required 
and  they  stated  that  technical  risks  during  the  production  phase 
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are  low.  However,  in  our  January  1983  report^  on  the  Army's 
Apache  helicopter  and  Hellfire  missile  programs,  we  reported 
that  there  were  still  significant  shortcomings  in  some  major 
Hellfire  components.  Because  the  Army  decided  not  to  conduct 
the  technical  analyses,  the  risks  associated  with  these 
shortcomings  were  not  considered  in  the  Hellfire  production 
estimate . 

Another  example  of  insufficient  consideration  for  risk 
relates  to  the  Apache  helicopter  dicussed  on  page  23. 

Consistency  of  inflation  recognition 

The  cost  estimates  for  weapon  systems  must  be  consistent 
for  the  Congress  to  make  comparisons  between  them.  DOD  often 
did  not  recognize  inflation  consistently  in  the  weapon  systems 
we  reviewed.  For  example,  in  some  cases  we  found  (1)  contractor 
inflation  indexes  were  used  to  calculate  inflation, 

(2)  different  inflation  indexes  were  used  for  estimates  of  the 
same  weapon  systems,  and  (3)  inflation  was  used  as  a  device  to 
hide  other  cost  increases. 

Use  of  contractor  indexes 


DOD  cost  estimates  are  required  to  use  the  Office  of 
Management  and  Budget  (0MB) -mandated  inflation  rates — the 
December  1980  SAR  estimate  for  the  Bradley  did  not.  The 
December  1980  SAR  (fiscal  year  1982  approved  program)  for  the 
Bradley  Fighting  Vehicle  System  reported  a  $2,748  million 
increase  in  the  baseline  vehicle  estimate  over  the  September 
1980  SAR  (fiscal  year  1981  approved  program).  According  to 
program  officials,  most  of  this  increase,  $2,541  million, 
reflected  contractor  projections  of  inflation.  These  projec¬ 
tions  were  significantly  higher  than  the  0MB  mandated  indexes 
that  were  supposed  to  be  used. 

Different  indexes  used  on  same  program 

The  August  1981  program  office  estimate  and  the  independent 
estimate  for  the  Army's  Hellfire  used  different  OSD/OMB  infla¬ 
tion  indexes.  The  program  office  estimate  used  March  1981 
inflation  indexes  but  the  independent  estimate  used  July  1981 
indexes.  Since  the  July  indexes  projected  lower  inflation  fac¬ 
tors  than  the  March  ones,  the  independent  estimate  was  lower 
than  it  would  have  been  otherwise.  The  result  of  using  differ¬ 
ent  inflation  indexes  was  that  the  difference  between  the  two 
estimates  was  about  $55  million  less  than  it  would  have  been  if 
the  same  index  had  been  used  for  both  estimates. 


^The  Army's  AH-64  Helicopter  and  Hellfire  Missile  Retain  Risks 

as  They  Enter  Production.  (Unclassified  digest.) 
(GAO-C-MASAD-83-9,  Jan.  26,  1983.) 
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Inflation  used  to  hide  other  costs 


The  treatment  of  inflation  and  the  practice  of  some  program 
managers  to  attribute  cost  increases  to  inflation  was  discussed 
in  an  April  1982  Comptroller  of  the  Army  report.  The  report 
states : 

"Program  Managers  tend  to  define  inflation  in  differ¬ 
ent  terms.  Often  any  change  other  than  a  programmatic 
change  that  occurs  in  a  system  is  attributed  to  infla¬ 
tion.  As  an  example,  the  Pershing  II  Program  Manager 
has  developed  inflation  indices  based  on  contractor 
labor  rates  and  material  prices  to  account  for  ' infla¬ 
tion'  in  the  Pershing  II  program.  This  is  in  direct 
contradiction  to  the  definition  of  inflation  described 
in  [DOD  7000.3  G] .  For  cost  estimating,  programming, 
budgeting,  and  reporting,  the  Army  must  employ  a  defi¬ 
nition  of  inflation  based  on  national  statistics." 

According  to  program  office  officials  and  the  Secretary  of 
Defense  Decision  Memorandum  issued  in  conjuction  with  the  pro¬ 
duction  decision,  the  Army's  Apache  helicopter  production  esti¬ 
mate  of  March  1982  included  an  adjustment  of  $72.6  million  to 
conform  to  the  President's  budget.  Strict  application  of  0MB 's 
January  1982  inflation  index  would  have  resulted  in  a  production 
cost  estimate  of  $7,453.4  million — the  estimate  required  to  con¬ 
form  to  the  President's  budget  was  $7,380.8  million.  This  $72.6 
million  adjustment  was  explained  to  us  as  an  inflation  adjust¬ 
ment.  However,  the  program  office  accounted  for  this  change  by 
reducing  the  base  program  by  $56.5  million  in  recurring  produc¬ 
tion  costs,  $16  million  in  the  costs  for  spares,  and  $.2  million 
in  other  costs.  In  addition,  in  the  SAR  report,  this  reduction 
is  spread  among  the  various  cost  variance  categories — it  cannot 
be  specifically  identified. 

The  Army's  Bradley  System  showed  a  $2.5  billion  increase 
because  contractor  indexes  were  used  in  calculating  inflation  in 
its  December  1980  SAR.  (See  p.  15.)  A  subsequent  cost  analysis 
by  the  program  office  indicates  that  the  increase  was  not  purely 
inflation,  but  included  some  real  cost  growth. 


Cost  per  vehicle 


$543,000  to  $639,000  =  17.7% 
$639,000  to  $818,000  =  32.9% 
$543,000  to  $818,000  =  50.6% 


Real 

Inflation 

Total 


The  real  cost  increase  per  vehicle  is  $96,000  ($639,000  minus 
$543,000).  The  total  real  cost  growth  for  6,882  vehicles  is, 
therefore,  $660.7  million.  This  increase  was  included  in  the 
$2.5  billion  inflation  increase  (calculated  using  contractor 
indexes)  reported  in  the  December  1980  SAR.  The  program  office 


16 


analysis  pertained  to  rollaway  cost,  which  excludes  some  major 
program  components;  therefore,  the  figures  in  the  above  chart  do 
not  explain  the  entire  $2.5  billion  increase. 

IMPROVEMENTS  NEEDED  IN  THE  DIRECTION 
PROVIDED  BY  DOD  MANAGEMENT 


Cost  estimates  are  not  made  in  a  vacuum.  DOD  management 
officials  tell  estimators  what  to  estimate  and  how,  in  terms  of 
the  physical  and  performance  characteristics  of  the  system  and 
the  techniques  and  assumptions  to  be  used.  Once  a  cost  estima¬ 
tor  has  prepared  the  figures  on  what  a  system  should  cost,  based 
on  the  system  definition  and  program  assumptions  provided,  the 
estimate  must  progress  through  the  successive  layers  of  DOD 
management.  One  problem  has  been  that  such  a  hierarchical  orga¬ 
nization  tends  to  influence  estimators  to  use  the  most  optimis¬ 
tic  assumptions  possible  because  they  result  in  lower  estimates 
which  are  seen  as  making  a  greater  contribution  to  program 
approval . 

This  problem  has  been  addressed  in  past  studies,  including 
The  Air  Force  Cost  Estimating  Process;  The  Agencies  Involved 

and  Estimating  Techniques  Used,  (see  app.  VI  item  1)  and  Defense 

Acquisition;  A  Game  of  Liars  Dice?  (see  app.  VI  item  7). 

When  estimators  are  encouraged  to  use  optimistic  assump¬ 
tions  regarding  system  design  requirements,  number  of  units, 
length  of  time  for  procurement,  financial  considerations  regard¬ 
ing  contractors,  and  so  forth,  it  results  in  an  overly  optimis¬ 
tic  estimate  of  what  the  system  could  cost  if  the  acquisition 
strategy  goes  perfectly.  However,  history  shows  that  system 
acquisitions  rarely  go  according  to  plan — a  myriad  of  influences 
determine  their  ultimate  cost  and  performance.  These  influences 
have  resulted  in  systems  exceeding  original  and  revised  cost 
estimates . 

Management  direction  is  often  optimistic 

On  March  22,  1982,  the  Assistant  Secretary  of  the  Navy  for 
Shipbuilding  and  Logistics  issued  guidance  that  the  ship  con¬ 
struction  budget  for  fiscal  year  1984  be  repriced  using  less 
conservative  estimating  assumptions  to  produce  a  more  optimistic 
estimate.  Although  the  guidelines  of  the  March  22  memorandum 
were  not  implemented,  the  Navy  did  develop  more  optimistic 
estimating  assumptions.  These  assumptions  were  incorporated  in 
the  April  9  guidelines  used  in  the  subsequent  repricing  of,  and 
attendant  $2.7  billion  reduction  in,  the  Navy's  Five-Year  Ship¬ 
building  Plan. 

In  the  March  22  memorandum,  the  Assistant  Secretary  stated 
that  high  estimates  are  a  self-fulfilling  prophecy — estimates 
should  be  kept  low  to  force  constraints  on  contractors.  Among 
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the  major  cost  driving  assumptions  that  were  to  be  reestimated 
optimistically,  were: 

— Assume  that  the  shipyard  with  the  lowest  labor  rates  will 
receive  the  contract  award  where  more  than  one  yard  is 
competing . 

— Assume  no  growth  beyond  target  cost  where  a  50/50  share 
line^  for  cost  overruns  exists. 

— Eliminate  all  future  characteristic  changes  (this  is  a 
budgeted  reserve  for  changes  to  ship  design  and  technical 
specifications) . 

— For  competitive  procurements,  assume  10  percent  maximum 
profit. 

— Eliminate  change  order  margins  or  identify  magnitude  and 
rationale.  Change  order  line  for  repeat  construction 
must  be  less  than  for  prior  years.  Do  not  use  percentage 
of  basic  construction  costs  to  calculate  change  order 
allowances,  but  reduce  the  amount  to  no  more  than  the 
absolute  value  of  the  first  program  year,  escalated  to 
the  current  year. 

— Shorten  all  construction  periods  1  month;  for  50/50  share 
lines  reduce  the  period  by  2  months. 

— Reduce  labor  hours  and  material  costs  by  10  percent. 

— Eliminate  program  managers'  growth*^  as  a  percentage  of 
government-furnished  equipment  costs.  Substitute  an 
absolute  value  equal  to  the  first  program  year  and  esca¬ 
late  to  the  current  year. 

These  guidelines  were  not  implemented.  Officials  of  the 
Naval  Sea  Systems  Command  determined  the  March  1982  guidelines 
to  be  overly  optimistic  and  on  April  9,  1982,  the  guidelines 
were  revised  as  follows: 

— Where  competition  is  expected,  use  the  cost  factors  sup¬ 
porting  the  lowest  cost  estimate  when  preparing  the  Pro¬ 
gram  Objective  Memorandum. 


^Any  cost  overruns  beyond  target  cost  are  to  be  shared  equally 
between  the  government  and  the  contractor. 

^A  funding  category  for  unforeseen  contigencies. 
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— No  guideline  for  growth  beyond  target  cost  was  estab¬ 
lished  because  growth  was  never  assumed  beyond  the  target 
cost . 

— The  future  characteristics  changes  should  be  specifi¬ 
cally  determined  by  the  Chief  of  Naval  Operations  sponsor 
with  a  specific  change  in  mind. 

— For  situations  where  real  opportunity  for  competition 
exists,  estimates  should  include  a  10-percent  profit  fac¬ 
tor  . 

— Ten  percent  of  the  basic  ship  cost  and  planning  cost  is 
allowed  for  the  lead  ship  change  order  line.  Five  per¬ 
cent  of  the  basic  cost  is  allowed  for  follow-on  ship 
change  order  lines. 

— Scheduled  construction  times  for  planning  will  be  deter¬ 
mined  by  the  office  of  the  Naval  Sea  Systems  Command 
Principal  Deputy  Commander  for  Acquisition.  Ships  in 
fiscal  year  1985  and  beyond  will  be  scheduled  considering 
the  source  time  factors  as  used  for  the  fiscal  year  1984 
line  where  each  ship's  time  was  reduced  at  least  1  month. 

— The  March  22,  1982,  direction  to  reduce  labor  hours  and 
materials  by  10  percent  was  considered  unrealistic. 

— The  allowable  amount  for  program  manager's  growth  was  set 
at  10  percent  of  government-furnished  equipment  costs. 

The  guidelines  in  the  March  22,  1982,  Navy  memorandum  are 
an  example  of  optimistic  direction  by  higher  management. 

Although  they  were  not  implemented,  the  excessively  optimistic 
March  22  guidelines  moved  the  Navy  toward  more  optimistic 
assumptions.  This  resulted  in  the  guidelines  issued  on  April  9. 
The  optimistic  character  of  the  April  9  guidelines  is  shown  by 
the  $2.7  billion  reduction  in  the  subsequent  repricing  of  the 
Navy's  Five-Year  Shipbuilding  Plan.  Other  examples  of  optimis¬ 
tic  direction  are 

— use  of  optimistic  contractor  data, 

— forcing  cost  estimates  to  conform  to  the  service  budgets, 
and 

— the  exclusion  of  relevant  program  costs. 

Contractor  cost  data  sometimes 
inaccurate  or  optimistic 

DOD  cost  estimators  told  us  the  competitive  nature  of 
business  drives  optimistic  contractor  estimates.  They  also  told 
us  contractors  may  underbid  to  "sell"  a  program  to  the  service 
and  the  Congress.  Program  offices  sometimes  base  their 
estimates  on  this  overly  optimistic  contractor  data. 
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This  problem  has  been  discussed  in  a  number  of  past  publi¬ 
cations,  including  the  report  of  the  Special  Panel  on  Defense 
Procurement  Procedures,  House  Committee  on  Armed  Services  (see 
app.  VI  item  4),  A  Game  of  Liar's  Dice  by  Dr.  Walter  B.  Laberge 
(see  app.  VI  item  7),  Cost  Overruns  in  Major  Weapon  Systems: 
Current  Dimensions  of  a  Longstanding  Problem  by  the  Congression¬ 
al  Research  Service  (see  app.  VI  item  11),  and  our  report.  Cost 
Growth  in  Major  Weapon  Systems  (see  app.  VII  item  4). 

Two  of  the  weapon  systems  in  our  sample  provide  illustra¬ 
tions  of  this  type  of  problem — the  Hellfire  missile  program  and 
the  Apache  helicopter  program. 

The  Hellfire  cost  estimates  were  based  on  the  March  1981 
contractor  design- to-unit  production  cost  data.  The  data  was 
used  even  though  the  project  office  did  not  consider  that  data 
to  be  reliable  because  of  the  contractor's  history  of 
understated  cost  reports — the  contractor's  missile  cost  goal 
increased  from  $3,448  to  $5,710  per  unit  between  October  1976 
and  March  1981. 

On  the  Apache  helicopter  program,  the  accuracy  of  the  con¬ 
tractor's  (Hughes)  cost  data  also  appears  questionable. 

Problems  and  inaccuracies  with  Hughes'  cost  data  were  cited  by 
representatives  of  the  Army  Comptroller,  the  program  office,  the 
Defense  Contract  Audit  Agency,  and  Hughes.  The  contractor  cost 
data  the  Apache  program  office  used  to  prepare  its  February  1980 
cost  estimate  was  optimistic  and  inacurate.  The  $1.55  billion 
cost  increase  in  the  September  1981  estimate  was  attributed 
largely  to  higher  contractor  costs. 

Army  Comptroller  and  program  office  representatives  told  us 
that  Hughes  lacked  experience  in  building  a  complex  aircraft 
like  the  Apache.  Hughes'  cost  data  base  is  built  on  industry 
standards  reportedly  adjusted  for  Hughes'  experience  on  its  com¬ 
mercial  product  line.  Army  representatives  said,  however,  that 
Hughes'  data  base  was  inaccurate  and  did  not  reflect  actual 
experience.  This  resulted  in  unrealistically  low  cost  esti¬ 
mates.  A  Hughes  spokesperson  stated  that  Hughes  was  optimistic 
in  its  early  estimating.  Hughes  hired  new  management  officials 
who  recognized  the  problems  and  adjusted  the  data  base,  which 
resulted  in  a  large  increase  in  Hughes'  labor  hour  and  cost 
estimates  as  reflected  in  the  September  1981  program  office 
estimate.  Although  Hughes  representatives  said  the  company  had 
taken  corrective  actions.  Defense  Contract  Audit  Agency  repre¬ 
sentatives  told  us  that  the  practices  still  exist. 

Program  cost  estimates  are  forced  to 

conform  to  the  budget 

In  some  cases,  weapon  system  cost  estimates  are  forced  to 
fit  under  the  fiscal  constraints  of  the  DOD  budget  by  arbitrari¬ 
ly  reducing  the  estimate.  The  services'  total  program  cost 
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estimates  are  prepared,  reviewed,  and  validated  under  a  ceiling; 
according  to  DOD  Instruction  7000.3,  the  cost  estimate  appearing 
on  the  December  SAR  must  correspond  to  the  President's  budget. 
One  example  of  the  budget  constraint  on  an  estimator's  best 
estimate  is  the  fiscal  year  1984  repricing  directive  for  Navy 
shipbuilding  explained  previously.  Another  example  identified 
during  our  review  occurred  in  March  1982  when  the  DSARC  produc¬ 
tion  review  on  the  Apache  helicopter  reduced  the  estimate  by 
$72.6  million  to  conform  to  the  fiscal  year  1983  budget  guid¬ 
ance.  To  achieve  the  lowest  estimate  possible,  the  most 
optimistic  assumptions  are  used  regarding  technical,  risk, 
development,  and  production  problems,  and  schedule  and  design 
perturbations. 

As  illustrated  by  the  example  of  the  Apache  helicopter,  the 
budget  process  sometimes  forces  estimates  down  artifically  to 
get  all  programs  approved.  According  to  DOD  officials,  in  this 
environment  if  tasks  scheduled  for  the  current  fiscal  year  can¬ 
not  be  paid  for  in  the  current  budget,  those  tasks  are  postponed 
until  future  years.  Similarly,  the  effects  of  problems  in 
developing  the  system,  requirements  changes,  funding  and  sche¬ 
dule  changes,  increases  in  units,  and  so  forth,  which  cannot  be 
funded  in  the  current  budget  are  postponed  to  future  budget 
years — the  "out  years" — which  are  essentially  unconstrained. 
These  tasks  tend  to  be  separate  from  the  budgeted  costs  initial¬ 
ly  associated  with  them  in  a  phenomenon  known  as  "requirements 
uncoupling" . 5  The  result  has  been  significant  cost  growth  when 
the  "out  years"  become  the  current  year  and  the  postponed  costs 
begin  to  exceed  the  budget  projections. 

Relevant  program  costs  are  excluded 

from  cost  estimates 


Relevant  program  costs  are  often  excluded  from  systems' 
cost  estimates  by  omitting  costs  associated  with  changes  to  the 
original  requirements.  Examples  of  these  excluded  costs  were 
identified  in  the  following  systems; 

— The  September  1979  program  office  estimate  for  the  Brad¬ 
ley  System  excluded  product  improvements  specified  in  the 
June  1979  approved  system  requirement.  These  improve¬ 
ments  added  $740.9  million  to  the  program  1  year  later. 

— Costs  for  a  seeker  to  meet  an  Army  requirement  for 

the  Hellfire  missile  were  excluded  from  the  systems'  cost 
estimates.  Projections  for  those  costs  range  up  to  $1.1 
billion.  In  addition,  the  program  office  and  the 
Department  of  the  Army  disagree  over  the  estimates  for 
the  missiles'  storage  facilities  in  Europe. 


5w.  M.  Allen,  Causes  of  Weapons  Cost  Growth;  Three 
Perspectives ,  Resource  Management  Journal,  spring  1982. 
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— The  September  1981  program  office  production  estimate  for 
the  Apache  helicopter  did  not  include  $291.4  million  for 
combat  mission  simulators  or  $59  million  in  aircraft 
survivability  equipment.  The  July  1982  estimate 
included  the  survivability  equipment. 

Force  objectives  for  some 

estimates  are  undecided 


The  cost  estimates  reported  to  the  Congress  are  suppose  to 
represent  the  cost  of  the  entire  weapon  system  program.  Some  of 
those  estimates,  however,  are  reporting  costs  based  on  total 
numbers  of  units  that  are  different  from  the  numbers  used  in 
internal  DOD  force  projections.  Examples  of  systems  where  total 
force  requirements  are  undecided  are; 

— The  Army  has  not  decided  on  procurement  quantities 

for  the  Hellfire.  After  the  August  1981  program  office 
estimate  was  made,  the  planned  quantity  increased  from 
24,600  to  35,756  units.  The  Army  has  increased  this  to 
42,332  missiles  in  December  1982  and  to  48,696  in 
December  1983.  Project  officials  told  us  they  do  not 
know  how  many  units  will  be  eventually  procured.  They 
indicated  the  number  of  Hellfire  missiles  will  probably 
increase  by  6,000  each  year  as  long  as  the  budget  allows. 

— The  September  1979  SAR  for  the  Bradley  System  showed 
9,261  vehicles  at  a  cost  of  $6.5  billion.  At  the  same 
time  the  September  1979  program  office  estimate  for  the 
Bradley  showed  6,882  vehicles  at  a  cost  of  $7.7  billion. 

— The  number  of  Navy  LSD-4  Is  has  ranged  from  3  to  16  in 
the  period  from  the  lead-ship  authorization  in  1981  until 
now.  Only  10  ships  are  reflected  in  the  Navy's  December 
1983  SAR,  6  ships  less  than  indicated  by  internal  plan¬ 
ning  documents. 

FULLER  USE  OF  INDEPENDENT  COST  ESTIMATORS' 

RECOMMENDATIONS  WOULD  MAKE  COST  ESTIMATES 
MORE  RELIABLE  AND  VALID 


Although  OSD  and  the  services  have  established  independent 
cost  estimating  groups  to  help  ensure  that  cost  estimates  are 
more  reliable  and  valid,  efforts  by  such  groups  have  not  always 
been  effective.  We  found  some  independent  cost  estimators' 
estimates  did  not  appear  to  be  fully  considered  by  DOD  decision¬ 
makers  and  their  recommendations  were  not  accepted. 

Recently  the  Congress  has  placed  increased  emphasis  on  the 
use  of  independent  estimates.  The  1984  DOD  Authorization  Act 
states 

"the  Secretary  of  Defense  may  not  approve  the 
full-scale  engineering  development  or  the  production 
and  deployment  of  a  major  defense  acquisition  program 
unless  an  independent  estimate  of  the  cost  of  the 
program  first  has  been  submitted  to  (and  considered 
by)  the  Secretary  of  Defense." 
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The  Secretary  is  also  required  by  May  1,  1984,  to  submit  a 
written  report  to  the  Congress  on  the  use  of  independent 
estimates  in  the  planning,  programming,  budgeting,  and  selection 
process  for  major  defense  acquisition  programs.  These 
congressional  actions  may  encourage  solutions  to  some  of  the 
problems  outlined  below.  Progress  will  be  shown  when  program 
cost  estimates  include  the  recommendations  of  independent 
estimators  and  the  resulting  estimates  are  more  accurate. 

Army's  Apache  helicopter 

The  independent  cost  estimators'  estimate  and  recommenda¬ 
tions  for  the  Apache  program  production  decision  were  not  fully 
accepted  by  ASARC  and  DSARC. 

The  independent  cost  estimating  team^  determined  that  the 
Apache  estimate  should  be  $576  million  higher  than  the  program 
office  estimate,  and  also,  that  an  additional  $562.9  million  in 
risk  money  should  be  added  for  production  risks  unique  to  this 
program. 

The  Comptroller  of  the  Army  accepted  the  program  office 
estimating  methodology  and  the  lower  program  office  estimate, 
but  recommended  to  the  ASARC  that  the  risk  money  be  added  to  the 
program.  ASARC  also  accepted  the  program  office  estimate  and 
the  recommendation  for  risk  money  but  reduced  the  production 
quantities  from  536  aircraft  to  446  aircraft,  and  reduced  the 
risk  money  from  $562.9  million  to  $528.1  million.  In  January 
1982,  the  Apache  office  prepared  an  amended  estimate  to  reflect 
the  ASARC  approved  program  quantity  and  risk  money. 

The  OSD  Cost  Analysis  Improvement  Group  then  recommended 
approval  of  the  amended  program  estimate  to  DSARC.  DSARC  and 
the  subsequent  Secretary  of  Defense  decision  essentially 
approved  the  ASARC  recommended  production  quantity  and  risk 
money,  but  reduced  the  total  cost  estimate  by  $72.6  million  to 
agree  with  the  fiscal  year  1983  budget. 

Although  the  production  estimating  and  decision  process 
identified  production  risk  and  set  aside  money  for  it,  the  pro¬ 
duction  risk  money  did  not  last  long.  The  program  was  restruc¬ 
tured  to  use  the  risk  money  to  buy  69  more  aircraft  for  a  total 
production  quantity  of  515,  thereby  effectively  doing  away  with 
the  independent  estimator's  recommendation.  This  decision  to 
delete  the  risk  funding  provisions  was  made  even  though  the 
production  risk  situation  had  not  changed  between  time  of  the 
ASARC  decision  and  the  decision  to  delete  the  risk  funding. 


^This  team  consisted  of  representatives  from  the  Office  of  the 
Comptroller  of  the  Army,  the  Army  Materiel  Development  and 
Readiness  Command,  and  the  Army  Aviation  Research  and 
Development  Command. 
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Army's  Bradley  Fighting  Vehicle  System 

An  OSD  analysis  of  the  September  1979  Bradley  estimate 
pointed  out  that  the  fire  control  portion  of  the  estimate  could 
be  understated  by  as  much  as  30  percent.  No  change  was  made  to 
the  estimate.  The  next  year  the  Hughes  Aircraft  Company's  esti¬ 
mate  for  the  fire  control  system  rose  significantly — from  $597.8 
million  to  $934  million. 

Air  Force  B-1B  Bomber 


On  November  1,  1981,  the  Cost  Analysis  Improvement  Group 
(CAIG)  issued  a  report  on  the  B-IB  in  which  it  recommended  the 
Air  Force  add  allowances  to  its  estimate  to  provide  for  a  full 
cruise  missile  capability,  not  include  multiyear  contracting 
savings  until  actual  contractor  bids  are  in  hand,  and  add  allow¬ 
ances  for  simulators.  The  Cost  Analysis  Improvement  Group 
recommendation  on  cruise  missile  capability  was  adopted  by  the 
Air  Force  resulting  in  a  total  program  estimate  of  $20.5 
billion.  The  $1.1  billion  provisions  regarding  multiyear 
contracting  and  simulators  were  not  adopted. 

COST  DISCIPLINE  NEEDS 

TO  BE  ENFORCED 

SARs  show  that  total  program  costs  increase  year  after 
year — indicating  that  total  cost  is  not  a  real  constraint  on 
weapon  system  programs.  The  December  1983  SAR  reports  show  that 
the  costs  for  the  73  major  weapon  systems  have  grown  from  $339.8 
billion  to  $618.5  billion.  Appendix  XI  shows  the  average 
increases  in  program  cost,  excluding  inflation,  for  36  produc¬ 
tion  programs  included  in  the  December  1982  SARS,  are  $13  mil¬ 
lion  per  month  before  production  and  $18  million  per  month  after 
production.  These  increases  in  total  program  costs  have  become 
the  accepted  pattern — no  one  is  held  accountable  for  them.  Cost 
estimates  have  represented  a  threshold  that  is  broken  year  after 
year  without  significant  consequences  to  DOD,  the  services,  or 
the  weapon  system  programs. 

As  discussed  on  page  21,  it  is  the  budget  that  is  the  main 
program  constraint.  Tasks  not  affordable  in  the  current  budget 
year  are  postponed  to  future  years  ("out  years") — which  are 
essentially  unconstrained. 

Examples  drawn  from  our  selected  weapon  systems  illustrate 
that  the  cost  estimates  presented  at  the  milestone  decision 
points  are  not  effective  cost  constraints.  On  the  Bradley,  the 
estimate  used  for  the  production  milestone  decision,  scheduled 
for  January  1980,  showed  a  total  program  cost  of  $7,742  mil¬ 
lion.  Eleven  months  later  the  December  1980  SAR  showed  a  total 
program  cost  of  $12,686.7  million.  On  the  Apache  the  full-scale 
development  milestone  estimate  (Milestone  II)  showed  a  total 
cost  of  $3,758.1  million  for  536  aircraft.  The  Apache  produc¬ 
tion  milestone  decision  showed  a  total  cost  of  $7,380.8  million 
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for  446  aircraft.  The  Hellf ire's  estimate  for  the  full-scale 
development  decision  in  March  1976  was  $735  million.  In  March 
1982  the  production  milestone  estimate  was  $2,047.6  million. 

For  the  Navy's  LAMPS  MK  III  program,  the  cost  estimate  in  March 
1978  (full-scale  development)  was  $3,907.6  million;  in  June  1982 
(production  milestone)  the  estimate  was  $6,745.6  million;  and  in 
December  1982  the  estimate  was  $8,261.6  million. 

OSD  needs  to  enforce  cost  discipline  by  making  the  services 
more  accountable  for  staying  within  their  cost  estimates. 

Program  design  and  operational  requirements,  schedule,  numbers 
of  units,  economically  efficient  production  rates,  and  other 
assumptions  critical  to  cost  estimates  should  be  firmly 
established.  Changes  to  these  requirements  should  be  minimized, 
and  approved  only  on  the  basis  of  an  urgent  need,  and  only  after 
considering  the  cost  effect  of  such  changes.  These  estimates 
would  then  be  established  as  not-to-exceed  thresholds.  Any 
program  in  danger  of  breaching  this  threshold  would  be  rigorous¬ 
ly  evaluated  and  assessed  to  determine  whether  it  should  be 
restructured,  discontinued,  or  permitted  to  proceed  as  planned. 

CONCLUSIONS 


We  found  several  weaknesses  exist  in  the  guidance,  assump¬ 
tions,  methodologies,  and  data  used  to  develop  cost  estimates: 

— The  quality  and  completeness  of  cost  estimating  guidance 
varies  among  OSD  and  the  services.  Some  of  the  guidance 
is  too  general;  in  some  cases  it  is  conflicting. 

— Implementation  of  OSD  and  service  guidance  is  poor. 

This  leads  to  problems  of  inconsistent  estimate  structure 
and  content,  and  inadequate  estimate  documentation. 

— Cost  estimates  do  not  always  include  adequate  provisions 
for  program  risk.  Cost  estimators  seem  to  assume  that 
their  weapon  systems  will  not  suffer  from  the  risk 
factors  that  can  occur  on  weapon  programs. 

— Inflation  is  not  always  calculated  consistently. 

— Cost  estimators  are  sometimes  directed  to  use  optimistic 
assumptions  regarding  the  weapon  systems  to  be  esti¬ 
mated.  These  assumptions — system  design  requirements, 
number  of  units,  length  of  time  for  procurement,  finan¬ 
cial  considerations  regarding  the  contractors,  and  so 
forth — are  frequently  optimistic  when  compared  to  the 
experience  of  past  acquisitions. 

— Some  cost  estimates  are  based  on  contractor  data  which 
may  be  optimistic  and  could  lead  to  inaccurate  DOD  esti¬ 
mates. 
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Program  costs  are  constrained  by  the  program's  annual 
budget,  not  by  the  total  cost  estimates  established  at 
major  program  milestones,  which  hinders  cost  discipline 
and  accountability. 

Estimates  are  sometimes  kept  low  by  using  less  than  anti¬ 
cipated  numbers  of  units,  and  omitting  costs  associated 
with  changes  to  the  original  requirements. 

More  fully  considering  and  accepting  the  recommendations 
of  independent  estimators  would  make  DOD  cost  estimates 
more  valid  and  reliable. 

RECOMMENDATIONS 


We  recommend  that  the- Secretary  of  Defense  revise  OSD  and 
service  cost  estimating  guidance  to  ensure  that  it  is  clear  and 
consistent. 


The  Secretary  should  develop  measures  that  will  ensure  the 
cost  estimating  guidance  is  properly  implemented.  Appropriate 
implementing  of  guidelines  is  necessary  to  obtain  better  esti¬ 
mates.  We  also  recommend  that  the  Secretary  ensure  that 

all  costs  associated  with  a  weapon  system  are  included 
in  the  estimate, 

9ll  estimates  are  fully  documented, 

^PP^op^i^te  methodologies  are  used  to  develop  the  esti¬ 
mate,  and 

the  estimate  is  updated  when  significant  changes  occur  in 
the  program. 

In  addition,  the  Secretary  should  ensure  that: 

Program  risks  and  uncertainties  be  fully  identified  in 
DOD  cost  estimates.  Cost  estimators  should  be  directed 
to  structure  their  estimating  assumptions  to  consider 
proven  historical  cost  growth  drivers,  such  as  technical 
changes  due  to  engineering  problems  and  added  require¬ 
ments,  schedule  changes,  and  funding  instability. 

Inflation  should  be  calculated  consistently  and  in 
accordance  with  DOD  procedures. 

“^ost  estimates  be  based  on  realistic  assumptions  rather 
than  optimistic  assumptions. 
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— Cost  discipline  be  enforced  by  establishing  the  total 
program  acquisition  cost  estimate  developed  for  the  pro¬ 
duction  milestone  as  a  not-to-exceed  threshold.  A  pro¬ 
gram  in  danger  of  breaching  the  threshold  should  be 
assessed  to  determine  whether  it  should  be  restructured, 
discontinued,  or  permitted  to  proceed  as  planned. 

— DOD  management  make  fuller  use  of  independent  esti¬ 
mates.  The  independent  estimates  should  not  be 
arbitrarily  accepted,  but  the  recommendations  of  the 
independent  estimators  should  be  considered  and  decisions 
not  to  accept  them  should  be  fully  explained  and 
documented . 

We  believe  that  one  way  for  the  Secretary  to  address  these 
items  is  to  issue  a  cost  estimating  handbook.  This  handbook 
could  be  developed  by  the  OSD  Cost  Analysis  Improvement  Group 
along  the  lines  of  0MB' s  A-76  "Cost  Comparison  Handbook,"  or  as 
an  expansion  of  DOD's  1971  "Military  Equipment  Cost  Analysis" 
textbook.  The  handbook  should  be  provided  to  the  services'  cost 
estimating  functions,  and  would  ensure  that  consistency  and 
uniformity  among  the  services  exist  regarding  the  estimating 
procedures  used,  the  costs  to  be  included,  and  the  structure  of 
the  estimates.  It  should  also  provide  a  comprehensive  tracking 
and  documenting  system. 

In  developing  a  handbook,  the  following  elements  should  be 
included  as  part  of  the  required  documents  for  a  cost  estimate. 

— The  period  of  time  on  which  the  estimate  was  based  and 
whether  it  is  an  original  or  an  update  of  an  earlier 
estimate.  The  original  estimate  should  be  prepared  at 
the  time  approval  is  given  for  a  new  system.  Estimates 
should  be  updated  to  reflect  significant  changes  in  the 
program,  or  when  assumptions  used  to  develop  the  original 
estimate  change.  Program  costs  should  be  thoroughly 
reviewed  for  currency,  accuracy,  and  realism  at  each  of 
the  major  milestone  decision  points. 

— The  assumptions  the  cost  estimate  is  based  on,  particu¬ 
larly  the  system  design,  total  numbers  of  units  to  be 
procured,  production  rates,  and  inflation  rates.  Changes 
to  these  assumptions  should  be  shown  in  updated  estimates 
with  an  explanation  of  the  cost  impact  and  the  organiza¬ 
tional  authority  responsible  for  the  change. 

— Methodology  and  techniques  used  to  prepare  the  estimate, 
ranging  from  estimates  based  on  parametric  techniques  in 
the  early  stages  of  a  program,  to  detailed  estimates 
based  on  actual  experience  gained  to  date  in  the  later 
stages  of  development.  If  the  estimate  was  derived  from 
parametric  models,  the  type  of  model  should  be  fully 
described  along  with  its  relevance  to  the  current  system. 
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— Reasons  for  uncertainties  in  estimates  should  be  identi¬ 
fied,  strategies  laid  out  to  deal  with  them,  and  the 
range  of  cost  increases  that  could  occur  should  be  iden¬ 
tified.  Estimate  confidence  levels  should  be  stated 
along  with  a  sensitivity  risk  analysis  of  possible 
changes  in  requirements,  schedule,  and  quantity. 

The  estimate  should  contain  all  research  and  development, 
investment  and  procurement,  military  construction,  and 
operation  and  support  costs.  Any  excluded  costs  associ¬ 
ated  with  the  weapon  system,  but  included  under  another 
estimate,  should  be  fully  identified. 

Changes  to  DOD's  reporting  documents  required  by  the 
estimate . 

The  implementation  of  the  new  and  existing  guidelines 
should  be  monitored  by  the  Cost  Analysis  Improvement  Group,  as 
the  principal  advisor  within  DOD  concerning  cost  estimating. 

In  addition,  the  DOD  Inspector  General  could  periodically  review 
the  services'  efforts  to  implement  the  cost  estimating  guide¬ 
lines. 

DOD  COMMENTS  AND  OUR  RESPONSE 

The  following  summarizes  the  major  points  in  DOD's  comments 
regarding  this  chapter  and  provides  our  responses: 

DOD  comment 


DOD  did  not  agree  with  the  findings  contained  in  our  draft 
report  that  overall  DOD  guidance  and  service  guidance  did  not, 
in  some  cases,  meet  the  cost  estimating  criteria  which  we  artic¬ 
ulated  in  our  July  1972  report.  It  said  that  DOD  and  service 
directives  provide  more  comprehensive  guidance  than  is  needed  to 
meet  the  nine  criteria  cited  in  our  report.  (See  app.  IV.)  DOD 
went  on  to  cite  specific  DOD  and  service  directives,  regula¬ 
tions,  and  so  forth,  which  contained  this  guidance. 

DOD  further  stated  that  Army  Regulation  11-18  shows  that 
the  Army's  policy  is  to  include  all  relevant  costs  and  to  vali¬ 
date  cost  estimates. 

Our  reponse 

Our  1972  report  listed  nine  criteria  which  we  felt  were 
essential  for  effective  cost  estimating,  with  which  DOD  agreed. 

In  analyzing  DOD's  response  to  our  draft,  we  revised  the 
report  to  recognize  the  volumes  of  guidance  DOD  has  published  on 
cost  estimating.  We  do  not  make  a  point  of  DOD's  guidance  not 
meeting  the  criteria.  We  continue  to  believe,  that  the  examples 
provided  in  the  report  show  that  some  of  the  criteria  are  not 
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always  met  and  revisions  are  needed  in  DOD's  cost  estimating 
procedures. 

DOD  comment 


DOD  did  not  agree  with  our  findings  that  there  is  no  over¬ 
all  DOD  guidance  on  cost  estimating  and  that  the  services  have 
been  allowed  to  develop  their  own  guidance /  thus  compounding  the 
problem  further.  DOD  provided  us  with  a  long  list  of  over  30 
OSD  and  service  directives  which  they  said  were  adequate 
guidance . 

Our  response 

Although  our  draft  report  refered  to  the  lack  of  overall 
DOD  guidance,  we  revised  the  report  to  show  that  there  is  a  need 
to  improve  its  guidance. 

DOD  provided  an  impressive  list  of  service  regulations. 
Appendix  II  identifies  the  regulations  we  examined  during  this 
review. 

DOD  stated  that  a  major  function  of  the  Cost  Analysis 
Improvement  Group  and  its  counterparts  in  the  services  is  to 
ensure  that  all  cost  elements  are  included  in  the  cost  estimates 
they  review.  Our  example  on  page  11  regarding  the  Bradley 
illustrates  that  all  costs  are  not  included  in  the  cost 
estimates  the  Cost  Analysis  Improvement  Group  reviews.  Our 
example  on  page  24  regarding  the  B-IB  shows  that  the  costs 
included  in  the  estimate  the  Cost  Analysis  Improvement  Group 
reviewed  were  not  included  in  the  estimate  reported  to  the 
Congress. 

The  services  are  updating  their  cost  estimating  guidance. 
The  Navy  has  replaced  Naval  Material  Command  instruction 
(NAVMATINST)  7000. 19A  with  NAVMATINST  7000. 19B.  NAVMATINST 
7000. 19B  still  requires  cost  estimate  documentation  to  be  suffi¬ 
ciently  detailed  to  permit  the  reconstruction  of  the  cost  esti¬ 
mate  by  external  reviewers  and  refers  to  the  elements  of  infor¬ 
mation  specified  in  Chief  of  Naval  Operations  Instruction 
(OPNAVINST)  7000. 17A.  OPNAVINST  7000. 17A  describes  the  Cost 
Estimate  Documentation  Summary  discussed  on  page  12  of  this 
report.  Navy  cost  estimators  told  us  the  Navy  is  in  the  process 
of  updating  7000. 17A  to  eliminate  the  Navy  requirement  to  in¬ 
clude  in  one  document  most  of  the  information  a  reviewer  needs 
to  adequately  evaluate  a  cost  estimate. 

The  Army  is  in  the  process  of  initiating  some  major  changes 
to  its  cost  estimating  process;  but  these  changes  have  not  been 
incorporated  into  their  new  guidance.  The  Army  Materiel  Devel¬ 
opment  and  Readiness  Comand  Regulation  37-4  (dated  10/4/82) 
includes  nothing  on  total  risk  assessing  cost  estimates  (imple¬ 
mented  by  letters  of  instruction  dated  3/6/75  and  10/6/82), 
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annual  updates  for  estimates  (letter  dated  9/21/81),  and  joint 
independent  cost  estimating  teams  (Memorandum  of  Understanding 
dated  5/27/82).  The  Army  should  include  the  new  initiatives  in 
their  updated  guidance. 

The  Air  Force's  cost  estimating  handbook — AFSCM  173-1  dated 
April  17,  1972 — was  described  to  us  as  the  Air  Force's  primary 
cost  estimating  guidance.  It  is  no  longer  in  print.  The  Air 
Force  is  studying  how  to  update  it  and  will  not  print  more  until 
the  new  manual  is  available. 


DOD  stated  that  each  weapon  system  is  unique,  and  there  is 
no  single  approach  for  the  development  of  cost  estimates  that 
could  be  routinely  applied  to  all  systems.  We  agree  each  weapon 
system  is  unique  and  there  must  be  some  flexibility  in  cost 
estimating  guidance.  However,  DOD's  current  guidance  is  too 
general.  The  services'  guidance  is  much  more  specific  even 
though  each  service's  guidance  pertains  to  a  wide  variety  of 
weapon  systems.  The  Army,  for  example,  develops  cost  estimates 
for  rotary  aircraft,  tracked  vehicles,  and  missiles.  The  Navy 
develops  estimates  for  fixed-wing  aircraft,  rotary  aircraft, 
ships,  and  missiles.  The  Air  Force  develops  estimates  for 
fixed-wing  aircraft  and  missiles.  The  result  is  two  sets  of 
guidance  applied  to  estimating  costs  for  rotary  aircraft,  two 
sets  for  fixed-wing  aircraft,  and  three  sets  for  missiles.  it 
is  possible  for  OSD  to  develop  overall  guidance  and  it  should  do 
so.  DOD  said  its  policy  is  to  permit  the  services  to  tailor 
guidance  to  their  individual  needs. 


DOD  believes  the  current  guidance  provides  the  necessary 
framework  for  high  quality  estimates.  Our  examples  illustrate 
that  DOD  is  not  developing  high  quality  estimates  and  reporting 
them  to  the  Congress.  We  believe  improved  guidance  and  in¬ 
creased  monitoring  of  its  implementation  are  necessary  if  the 
problems  illustrated  by  our  examples  are  to  be  minimized  in  the 
future. 

DOD  comment 


DOD  partially  agreed  with  our  finding  that  it  did  not  docu¬ 
ment  data  sources,  assumptions,  methods,  and  decisions  basic  to 
the  estimate  for  virtually  every  system  we  reviewed.  it  did  not 
agree  with  our  statement  that  documentation  is  lacking  on  all 
weapon  systems. 

Our  response 

This  section  of  the  report  has  been  rewritten  to  recognize 
DOD  s  comments  and  to  more  clearly  present  what  we  found.  in 
1972  we  found  inadequate  documentation  on  virtually  all  systems 
reviewed.  On  the  current  assignment,  we  found  similar  problems 
on  three  of  the  systems  reviewed. 
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DOD  said  that  Cost  Analysis  Improvement  Group  reviews  have 
found  that  program  cost  estimate  documentation  has  generally 
been  adequate.  The  illustrations  drawn  ffom  the  seven  selected 
weapon  systems  show,  that  for  these  systems,  documentation  is 
not  adequate  for  determining  how  the  estimates  were  developed 
(see  pp.  11  to  13)  or  the  reasons  for  cost  growth  from  one 
estimate  to  the  next.  (See  p.  13.)  We  believe  cost  estimates 
cannot  be  used  to  establish  cost  discipline  if  costs  cannot  be 
traced  from  one  estimate  to  the  next  and  the  reasons  for  cost 
growth  cannot  be  identified. 

DOD  stated  that  we  did  not  ask  for  the  1978  LAMPS  estimate 
on  which  the  successive  iterative  estimates  were  based.  Our 
auditors  did  ask  repeatedly  for  the  1978  estimate,  but  were  told 
that  it  was  not  available.  DOD  provided  a  copy  of  the  computer 
printouts  available  for  the  1978  estimate  on  November  28,  1983, 
after  the  completion  of  our  review. 

DOD  comment 


DOD  did  not  agree  with  our  finding  that,  because  of  the 
competitive  nature  of  weapon  system  contracts,  contractors  may 
underbid  to  sell  a  program  to  DOD  and  that  cost  estimates  are 
often  based  on  this  overly  optimistic  contractor  data.  DOD 
stated  that  its  cost  estimators  use  a  variety  of  data  sources  in 
preparing  estimates  and  that  contractor  data  may  be  one  of  the 
sources  if  the  estimator  believes  the  data  should  be  used. 

Our  response 

The  examples  drawn  from  the  selected  weapon  systems  illus¬ 
trate  that  some  cost  estimates  are  based  on  optimistic  and  inac¬ 
curate  contractor  data.  (See  pp.  19  and  20.)  Our  examples  also 
show  that  the  independent  estimators  have  been  ineffective  in 
preventing  this  problem.  (See  p.  24.)  In  its  comment,  DOD 
describes  how  its  estimating  process  should  prevent  understated 
estimates.  Our  examples  show  the  system  did  not  work. 

DOD  comment 


DOD  partially  agreed  in  our  finding  that  several  weaknesses 
in  DOD  cost  estimates  should  be  corrected:  (1)  recognizing  the 
confidence  levels  associated  with  early  cost  estimates, 

(2)  ensuring  consistency  of  estimating  methodologies,  (3) 
identifying  program  risks,  and  (4)  ensuring  consistency  of 
inflation  recognition. 

DOD  stated  that  the  corrections  we  suggest  are  not  required 
because  guidance  is  already  in  place  on  the  first  three  and  DOD 
requires  the  use  of  uniform  inflation  rates.  With  regard  to 
risk,  DOD  stated  that  funds  provided  for  risk  in  the  budget  are 


the  first  to  be  cut  by  the  Congress  and  that  programs  cannot 
expand  beyond  their  yearly  budget  allocation  which  acts  as  a 
cost  constraint. 

Our  response 

DOD's  comments  indicate  that  DOD  already  has  policies  in 
place  to  address  the  problems  we  found.  We  acknowledge  the 
existence  of  the  documents  setting  forth  policy.  However,  the 
problems  we  found  occurred  despite  these  policies — more  needs  to 
be  done  to  ensure  implementation.  DOD  actions  to  improve  cost 
estimating  guidance  and  ensure  its  implementation  would  help 
prevent  the  cost  estimating  deficiencies  illustrated  by  the 
examples  in  this  report  from  occurring  in  the  future. 

DOD  strongly  disagreed  with  our  observation  that  the  annual 
budget  allocation  represents  the  only  cost  constraint  placed  on 
weapon  system  programs.  DOD  states  that  DOD  Directive  5000.1 
and  DOD  Instruction  5000.2  require  that  thresholds  be  estab¬ 
lished  for  performance,  schedule,  and  costs.  Breaching  these 
thresholds  requires  a  complete  new  DOD  review  of  the  program 
before  determining  whether  to  continue  with  its  acquisition. 

DOD  Directive  5000.1  and  DOD  Instruction  5000.2  do  not 
establish  firm,  not-to-exceed  thresholds  over  the  life  of  a  wea¬ 
pon  systems  acquisition  program,  and  do  not  require  a  decision 
on  program  continuation  when  a  cost  threshold  is  breached. 
Examples  drawn  from  our  seven  selected  weapon  systems  illustrate 
that  the  cost  estimates  presented  at  the  milestone  decision 
points  are  not  effective  cost  constraints.  On  the  Bradley,  the 
estimate  used  for  the  production  milestone  decision,  scheduled 
for  January  1980,  showed  a  total  program  cost  of  $7,742  mil¬ 
lion.  Eleven  months  later  the  December  1980  SAR  showed  a  total 
program  cost  of  $12,686.7  million.  On  the  Apache  the  full-scale 
development  milestone  estimate  (Milestone  II)  showed  a  total 
cost  of  $3,758.1  million  for  536  aircraft.  The  Apache  produc¬ 
tion  milestone  decision  showed  a  total  cost  of  $7,380.8  million 
for  446  aircraft.  The  Hellfire's  estimate  for  the  full-scale 
development  decision  in  March  1976  was  $735  million.  In  March 
1982  the  production  milestone  estimate  was  $2,047.6  million. 

For  the  Navy's  LAMPS  MK  III  program,  the  cost  estimate  in  March 
1978  (full-scale  development)  was  $3,907.6  million;  in  June  1982 
(production  milestone)  the  estimate  was  $6,745.6  million;  and  in 
December  1982  the  estimate  was  $8,261.6  million. 

In  addition,  cost  estimators  and  program  office  officials 
told  us  that  the  most  important  portion  of  the  program's  cost 
estimate  is  the  budget  year.  The  remainder  of  the  estimate  is 
of  value  for  planning  but  the  budget  year  is  the  portion  of  the 
estimate  that  demands  their  attention. 
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DOD  Directive  5000.1  (March  29,  1982)  requires  the 
establishment  of  a  dollar  threshold  at  Milestone  I  (concept 
selection)  that  cannot  be  exceeded  to  carry  the  program  through 
Milestone  II  (entry  into  full-scale  development).  DOD  Directive 
5000.1  does  not  require  a  similar  threshold  for  the  Milestone  II 
pifogram  go  ahead  decision  which  initiates  full-scale  development 
and  allows  the  program  to  proceed  to  production  and  deployment. 

DOD  Directive  5000.1  states  that  approval  to  proceed  into 
full-scale  development  or  into  production  shall  depend  on  the 
DOD  Component  demonstration  that  resources  are  available  or  can 
be  programmed  to  complete  development,  to  produce  efficiently, 
and  to  operate  and  support  the  deployed  system  effectively. 

Funding  availability  shall  be  reaffirmed  by  the  DOD  Component 
t)0fore  proceeding  into  production  and  deployment.  DOD  Directive 
5000.1  refers  to  DOD  Instruction  5000.2  for  the  specific  facets 
of  affordability  to  be  reviewed  at  milestone  decision  points. 

DOD  Instruction  5000.2  does  not  require  a  DSARC  review  be 
held  if  milestone  thresholds  are  breached.  It  requires  the 
Defense  Acquisition  Executive  be  notified  of  threshold  breaches 
to  decide  whether  or  not  a  program  review  or  another  DSARC  will 
be  required. 

DOD  Instruction  5000.2  also  states  that  a  cost-effectiveness 
analysis  for  all  major  acquisitions  shall  be  performed  by  the  DOD 
Components  to  support  Milestone  I  and  Milestone  II.  This  instruc¬ 
tion  points  out  that  a  Secretary  of  Defense  Decision  Memorandum 
documents  the  Secretary  of  Defense's  milestone  decision,  including 
approval  of  goals,  thresholds,  and  threshold  ranges  for  cost, 
schedule,  and  performance.  A  Secretary  of  Defense  Decision 
Memorandum  is  prepared  to  reflect  revised  thresholds  and  updated 
program  direction  resulting  from  thresh-old  breaches  or  projected 
breaches  reported  by  the  DOD  Compo-nent.  DOD  Instruction  5000.2 
also  states  that  programming  and  budgeting  decisions  that  may 
invalidate  a  milestone  decision  or  other  Secretary  of  Defense 
Decision  Memorandum  direction  shall  be  recommended  to  the  Defense 
Resources  Board  for  explicit  consideration  of  the  effect  on  mili¬ 
tary  capability  and  total  resource  requirements.  In  addition,  the 
DOD  Component  head  shall  explain  and  justify  to  the  Defense 
Resources  Board  differences  between  program  baselines  established 
at  Milestone  II  (entry  into  full-scale  development)  and  quantity 
and  funding  in  the  program  or  budget  under  review. 

DOD  comment 

DOD  did  not  agree  with  our  finding  that  assumptions  mandated 
by  DOD  management  in  preparing  cost  estimates  are  often  optimistic 
and  management  has  tended  to  pressure  estimators  through  control  of 
the  assumptions. 
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DOD  3lso  did  not  agree  that  physical  and  performance 
characteristics  provided  to  cost  estimators  produced  unrealisti¬ 
cally  low  estimates.  It  stated  that  cost  analysts  are  not  told 
how  to  do  their  analyses  and  are  not  forced  to  use  only  the  most 
optimistic  assumptions,  DOD  also  did  not  agree  with  our  example 
of  unrealistic  assumptions  presented  in  a  Navy  memorandum. 

DOD  particially  agreed  with  our  findings  that  decisions 
made  at  development  milestones  and  during  the  budget  process 
reinforces  the  over  optimism,  and  agreed  with  our  findings  that 
although  the  DOD  Acquisition  Improvement  Program  initiatives 
promise  long  needed  improvements,  they  still  face  resistance. 

Our  response 

The  relevant  portions  of  this  report  have  been  revised 
after  considering  DOD's  response  that  the  use  of  0MB  inflation 
rates  is  required  and  that  the  work  of  the  Naval  Sea  Systems 
Command  after  the  release  of  the  March  1982  memorandum  from  the 
Assistant  Secretary  of  the  Navy  for  Shipbuilding  and  Logistics 
resulted  in  the  April  9,  1982,  estimating  guidelines.  (See  pp. 
17  to  19.)  ^ 

DOD  states  that  its  estimates  are  based  on  formal  require¬ 
ments  documents  (such  as  the  Army's  Materiel  System  Requirements 
Specification)  which  do  not  provide  guidance  that  will  require 
the  cost  analyst  to  produce  an  unrealistically  low 
cost  estimate.  We  found  cases  where  the  DOD  estimates  did  not 
include  all  the  elements  specified  in  the  Materiel  System 
Requirements  Specification — the  Hellfire's  f ire-and-forget 
seeker  with  an  estimated  cost  in  excess  of  $1  billion  and  the 
Bradley  product  improvements  costing  $982.1  million  are  exam¬ 
ples. 


The  March  1982  memorandum  from  the  Assistant  Secretary  of 
the  Navy  for  Shipbuilding  and  Logistics  and  the  subsequent 
repricing  using  the  April  1982  Naval  Sea  Systems  Command  Con¬ 
troller  cost  estimating  guidelines  resulted  in  a  $2.7  billion 
reduction  in  the  Navy's  Five-Year  Shipbuilding  Plan,  When  asked 
why  the  new  costing  concepts  embodied  in  the  April  1982  guide¬ 
lines  were  not  used  earlier,  a  senior  Navy  cost  estimator  stated 
that  in  times  when  ship  costs  are  declining  it  is  much  easier  to 
use  more  optimistic  assumptions.  The  estimator  added  it  was 
very  difficult  to  "sell"  the  use  of  what  might  be  considered 
more  realistic  assumptions  in  such  situations.  When  asked  how 
much  of  the  $2.7  billion  savings  would  be  realized,  the  Navy 
estimator  could  only  comment  on  the  $432.9  million  pertaining  to 
fiscal  year  1984.  The  Navy  estimator  felt  that  of  the  $432.9 
million,  only  $115  million  relating  to  earlier  award  of  ship 
contracts  would  definitely  be  realized.  The  estimator  could 
offer  no  opinion  on  the  remainder ,  and  said  we  would  have  to 
wait  and  see  if  what  was  predicted  actually  became  reality. 
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The  examples  in  this  section  of  the  report  show  that  DOD 
cost  estimates  are  understated  as  a  result  of  the  direction  pro¬ 
vided  by  management  to  the  cost  estimators  or  by  decisions  made 
by  management.  DOD  states  that  when  budget  constraints  render 
given  programs  unaffordable,  these  programs  are  reexamined  and 
usually  restructured.  Our  point  is  that  the  cost  estimates  on 
which  the  budgets  are  based  are  sometimes  overly  optimistic. 

DOD  comment 


DOD  agreed  with  our  finding  that  independent  cost  estima¬ 
tor's  estimates  and  recommendations  are  not  always  accepted,  but 
that  its  concurrence  should  not  be  interpreted  to  mean  that 
independent  estimates  should  always  be  accepted. 

Our  response 

This  section  of  our  report  has  been  rewritten  to  indicate 
that  independent  estimates  should  not  be  arbitrarily  adopted  as 
the  official  program  estimates.  It  is  important,  however,  that 
recommendations  of  DOD ' s  independent  estimators  be  given  more 
consideration  by  DOD  management. 

DOD  comment  on  our  recommendation 


DOD  did  not  agree  with  our  proposal  to  establish  the  total 
program  acquisition  cost  estimate  developed  for  the  production 
milestone  as  a  not-to-exceed  threshold  saying  that  this  require¬ 
ment  is  already  established  under  DOD  Instruction  5000.2 
(Mar.  8,  1983). 

Our  response 

DOD  Instruction  5000.2  (Mar.  8,  1983)  does  not  establish  a 
not-to-exceed  threshold  at  any  milestone.  DOD  Directive  5000.1 
(Mar.  29,  1982)  requires  the  establishment  of  a  dollar  threshold 
at  Milestone  I  (concept  selection)  that  cannot  be  exceeded  to 
carry  the  program  through  Milestone  II  (entry  into  full  scale 
development) ,  but  has  no  similar  requirement  beyond  Milestone 
II. 


We  believe  a  not-to-exceed  threshold  should  be  established 
for  the  total  cost  of  weapon  systems  and  that  DOD  and  the  Con¬ 
gress  should  conduct  a  rigorous  review  of  any  weapon  system  in 
danger  of  breaching  the  threshold. 

DOD  comments  on  our  recommendation 


DOD  did  not  agree  with  our  recommendation  regarding  the 
issuance  of  standardized  criteria  for  cost  estimates.  DOD  con¬ 
tends  that  current  guidance  is  adequate  to  ensure  proper  service 
cost  estimates  and  to  allow  the  services  to  tailor  their  estima¬ 
ting  procedures  to  fit  their  needs. 
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Our 


response 


As  pointed  out  in  our  report  (see  pp.  9  and  10)  and  in  our 
response  to  DOD  s  previous  comments  (see  DOD  comment  on  pp.  28 
to  30),  we  believe  that  DOD  should  improve  its  guidance  as 
Xa  report.  A  "cost  estimating  handbook" 

from  th^  services  would  clarify  what  is  expected 

from  the  cost  estimating  process  and  would  help  resolve  the 

report””^  vagueness  and  conflicting  guidance  discussed  in  our 
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CHAPTER  3 


POD  REPORTS  TO  THE  CONGRESS 

NEED  MORE  REALISM 

POD  cost  estimates  are  reported  to  the  Congress  through 
SARs,  unit  cost  exception  reports,  and  the  budget  process. 

Major  concerns  have  been  expressed  by  the  Congress  and  its  over¬ 
sight  committees  for  some  time  over  the  accuracy,  timeliness, 
and  completeness  of  POP's  reporting,  particularly  in  the  area  of 
cost  estimating. 

In  its  review  of  weapon  systems  acquisition  procedures  and 
policies  of  the  POD,  the  Special  Panel  on  Defense  Procurement 
Procedures  of  the  House  Armed  Services  Committee  found  that  the 
SARs  do  not  identify  potential  problems  that  might  lead  to  cost 
growth.  Its  report  states  that  the  absence  of  timely  and 
complete  cost  information  that  more  clearly  identifies  potential 
cost  growth  hinders  the  efforts  of  the  Armed  Services  Committees 
in  exercising  legislative  and  oversight  responsibilities.  The 
Special  Panel  also  found  that  the  SARs  do  not  disclose 
substantive  changes  that  may  occur  in  a  program  during  the 
calendar  year.  The  Panel  noted  these  changes  then  appear  in  the 
December  SAR^ .  (See  app.  VI  item  4.) 

In  our  review  we  found 

— SARs  do  not  reflect  the  latest  anticipated  program 
acquisition  costs 

— SARs  do  not  show  total  acquisition  objectives  under 
consideration , 

—  important  cost  categories  are  not  reported  in  SARs, 

— costs  are  not  reported  consistently  in  SARs,  and 

— unit  cost  exception  reports  have  not  solved  the  problem 
of  a  lack  of  current  data  reported  to  the  Congress. 


^U.S.  Congress,  House  Committee  on  Armed  Services,  Special  Panel 
on  Defense  Procurement  Procedures  Report,  Weapons  Acquisition 
Policy  and  Procedures;  Curbing  Cost  Growth,  97th  Congress,  1st 
Session,  Committee  Print  No.  13,  February  12,  1982. 
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SARS  DO  NOT  REFLECT  THE  LATEST  ANTICIPATED 

PROGRAM  ACQUISITION  COSTS 


We  found  major  differences  between  the  officially  approved 
program  acquisition  cost  reported  in  SARs  and  the  currently 
anticipated  program  acquisition  cost  projected  by  the  program 
office.  These  differences  are  the  result  of  the  considerable 
time  lag  between  preparing  a  SAR  and  submitting  it  to  the  Con¬ 
gress,  and  the  requirement  that  SARs  reflect  the  President's 
approved  budget. 

DOD  states  that  it  is  not  required  to  report  program  costs 
that  are  not  part  of  the  officially  approved  program.  The  wea¬ 
pon  systems  we  reviewed  provided  examples  showing  that  costs 
considered  necessary  to  the  weapon  system  were  not  included  in 
the  official  program  and  therefore  not  reported  to  the  Congress. 
These  costs  were  sometimes  reported  after  a  period  of  several 
years.  Reporting  delays  of  this  kind  were  addressed  by  the 
Special  Panel  on  Defense  Procurement  Procedures  (see  app.  VI 
item  4)  and  led  to  a  permanent  requirement  for  the  unit  cost 
reports. 

Although  it  is  reasonable  that  DOD  should  have  the 
opportunity  to  evaluate  program  alternatives,  our  examples 
illustrate  that  in  some  cases  known  requirements  have  been 
excluded  from  program  cost  estimates.  In  some  of  these  cases, 
these  items  have  been  excluded  by  not  defining  them  as  part  of 
the  approved  program.  Specifics  from  the  programs  we  reviewed 
are : 


The  March/July  1982  program  office  estimate  for  the 
Bradley  System  projected  a  $429.4  million  cost  for  tube 
launched,  optically  tracked,  wire  guided  (TOW)  missile 
improvements.  The  TOW  requirement  was  also  included  in 
the  Bradley's  June  1979  Materiel  System  Requirement  Spe¬ 
cification — which  defines  the  scope  of  the  program. 

These  improvements  were  not  formally  approved  or  funded 
and  did  not  appear  in  the  SAR  estimates  until  December 
1983  when  they  were  shown  at  a  cost  of  $307.7  million. 

— The  December  1981  SAR  for  the  Apache  helicopter  reported 
a  cost  estimate  of  $72.6  million  lower  than  the  estimate 
resulting  from  the  November  1981  ASARC  production  review. 
The  March  1982  production  DSARC  lowered  the  ASARC 's 
approved  program  by  the  $72.6  million  to  agree  with  the 
fiscal  year  1983  budget.  The  March  and  June  1982  SARs 
reflected  this  lower  estimate.  Reporting  the  fiscal  year 
1983  budget  or  production  milestone  estimate  as  the 
approved  program  did  not  result  in  timely  and  informative 
reporting  to  the  Congress. 
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— February  and  July  1980  revisions  of  the  Hellfire  missile 
program  office  estimate  projected  procurement  costs  of 
$522.8  million  and  $553.9  million,  respectively.  The 
reporting  of  these  cost  increases  was  delayed  until  the 
December  1980  SAR.  This  caused  the  March,  June,  and 
September  1980  SARs  for  the  Hellfire,  which  were  report¬ 
ing  program  costs  of  $332.9  million,  to  be  understated  by 
as  much  as  $221  million. 

— Current  Hellfire  estimates,  including  the  December  1982 
SAR,  do  not  include  costs  for  a  new  missile  seeker  needed 
to  meet  Army  requirements  to  increase  the  survivability 
of  the  Apache/Hellf ire  system.  The  Army  canceled  an 
earlier  seeker  development  program,  estimated  at  over  $1 
billion,  and  is  evaluating  three  lower  cost  options  for 
the  seeker  requirement.  If  approved,  a  new  seeker  would 
further  increase  Hellfire  acquisition  costs.  Seeker 
development  costs  were  not  included  because  they  are  not 
part  of  the  officially  approved  program. 

The  Congressional  Budget  Office  found  similar  problems 
which  are  discussed  in  its  study  entitled  A  Review  of  the 
Department  of  Defense,  December  31,  1982  SARs.  (See  app.  VI 
item  10.)  The  study  found  over  $40  billion  in  costs  were  not 
included  in  13  DOD  weapon  systems  cost  estimates. 

The  Congress  has  addressed  this  problem  of  faulty  cost 
estimates  by  requiring,  in  one  case,  DOD  to  certify  to  the 
validity  of  its  cost  estimate.  This  occurred  on  the  B-IB  Bomber 
program. Although  this  report  discusses  some  weakness  in  the  B-IB 
esti-mate,  the  concept  of  certification  appears  sound. 

Expansion  of  this  practice  to  require  DOD  certification  on  all 
estimates  reported  to  the  Congress  may  be  of  value  in 
encouraging  DOD  to  include  all  costs  in  its  estimates. 

SARS  DO  NOT  SHOW  TOTAL  ACQUISITION 

OBJECTIVES  UNDER  CONSIDERATION 

As  discussed  in  chapter  2  (see  p.  22),  DOD  sometimes  does 
not  report  the  total  number  of  units  it  is  considering  for  its 
weapon  system  programs.  For  example,  the  LSD-41  Navy  decision 
coordinating  paper,  then  under  revision,  showed  16  ships.  The 
paper  directed  the  program  office  "to  continue  the  acquisition 
of  10  ships  while  planning  for  16." 

As  pointed  out  on  page  37,  the  Congress  is  concerned  about 
the  lack  of  complete  information  that  would  identify  potential 
cost  growth.  Footnoting  the  SARs  to  point  out  unit  increases 
being  considered  may  resolve  some  of  these  concerns. 
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IMPORTANT  COST  CATEGORIES  ARE 

NOT  REPORTED  IN  SARS 


DOD  Instruction  7000.3  requires  SAR  reports  to  include  only 
program  acquisition  costs^ — they  do  not  include  costs  such  as 
operation  and  support.  By  not  reporting  other  costs,  such  as 
operation  and  support,  a  significant  portion  of  program  life- 
cycle  costs  are  omitted  from  congressional  reporting.  These 
operation  and  support  costs  can  often  be  1/3  to  3/4  of  the  total 
life-cycle  costs  and  may  exceed  the  program's  production  cost. 
Because  these  costs  are  not  reported  in  SARs,  the  Congress  is 
not  aware  of  significant  cost  fluctuations.  For  example,  the 
1982  program  cost  estimate  for  the  Bradley  System  was  found  by 
the  Army  Audit  Agency  to  be  understated  by  $6.4  billion  in  oper¬ 
ation  and  support  costs. 

COSTS  ARE  NOT  REPORTED 

CONSISTENTLY  IN  SARS 


Some  costs  are  reported  in  one  SAR  only  to  be  dropped  from 
succeeding  SARs.  The  Bradley  System's  December  1980  SAR, 
included  a  $740.9  million  provision  for  product  improvements. 
These  product  improvements  were  never  formally  approved  and  were 
dropped  in  the  December  1982  SAR.  The  25MM  ammunition  procure¬ 
ment  program  for  the  Bradley  System  was  also  included  in  the  SAR 
from  December  1980  through  September  1982.  Program  costs  were 
reduced  by  $1.1  billion  when  ammunition  costs  were  deleted  from 
the  December  1982  SAR.  DOD  states  that  this  practice  is  allow¬ 
able  under  their  current  regulations. 

UNIT  COST  EXCEPTION  REPORTS  HAVE  NOT 

SOLVED  THE  PROBLEM  OF  A  LACK  OF  CURRENT 

DATA  REPORTED  TO  THE  CONGRESS 

The  1982  Defense  Authorization  Act  (Public  Law  97-86, 
section  917)  required  program  managers  of  major  defense  systems 
to  issue  unit  cost  reports  to  their  military  service  secretaries 
in  fiscal  year  1982.  The  law  provides  for  two  types  of  unit 
cost  reports — a  quarterly  unit  cost  report  and  an  exception  unit 
cost  report.  The  exception  report  is  issued  following  a  signi¬ 
ficant  increase  in  estimated  program  acquisition  unit  cost  or 
estimated  fiscal  year  procurement  unit  cost.  Public  Law  97-252 
extended  this  requirement  indefinitely. 


^Acquisition  costs  are  the  costs  of  developing  and  producing  the 
system;  and  life-cycle  costs  are  these  costs  plus  the  costs  of 
operating  and  supporting  the  system  over  its  service  life. 


40 


The  unit  cost  reports  were  intended  to  bring  attention  to 
unit  cost  increases  and  to  address  the  problem  of  reporting  cur¬ 
rent  data  regarding  DOD  program  cost  increases  to  the  Congress. 
The  Congress  was  concerned  that  information  reported  in  the  SARs 
was  primarily  historical  and  did  not  provide  timely  indications 
of  potential  cost  problems.  (See  app.  IV  item  4.)  The  unit 
cost  report  legislation  places  a  requirement  on  the  weapon 
systems  program  manager  to  file  a  report  when  there  is  reason  to 
believe  the  program  unit  costs  would  breach  thresholds  specified 
in  the  legislation.  The  Congress  expected  the  unit  cost  reports 
to  be  a  prospective  reporting  document  that  would  report  more 
current  data  than  the  SARs. 

However,  DOD  has  interpreted  this  requirement  in  a  manner 
that  does  not  allow  more  current  data  to  be  reported.  The  unit 
cost  report  regulation — DOD  Instruction  7220.31 — states  that 
unit  cost  reports  will  be  based  on  the  latest  estimate  of  the 
cost  of  the  current  DOD-approved  program.  This  is  the  same 
information  that  is  reported  in  the  SARs.  DOD  calculates  the 
difference  between  the  current  SAR  and  the  baseline  SAR  and 
reports  that  difference  in  the  unit  cost  report.  Therefore,  the 
unit  cost  reports  do  not  provide  a  more  prospective  view  of  wea¬ 
pon  system  costs  than  SARs.  They  report  changes  in  the  official 
DOD  program  only — not  cost  growth  that  is  anticipated  but 
defined  as  outside  the  approved  program. 

CONCLUSIONS 

The  reports  DOD  provided  to  the  Congress  need  more  realism. 
We  found  that; 

— Some  of  the  weapon  systems  cost  data  reported  to  the  Con¬ 
gress  is  not  accurate,  complete,  or  timely.  SARs  do  not 
always  report  the  most  currently  anticipated  program 
costs,  sometimes  report  certain  costs  inconsistently,  and 
do  not  report  operation  and  support  costs. 

— Cost  known  to  be  part  of  a  system's  requirements  are  not 
always  included  in  the  officially  approved  program  for 
the  system. 

— One  of  the  primary  reasons  the  unit  cost  exception 
reports  were  introduced  in  1982  was  the  Congress'  con¬ 
cern  that  the  cost  data  in  SARs  was  not  current. 

However,  since  DOD  has  based  the  unit  cost  exception 
reports  on  SARs,  the  data  they  contain  suffers  from  the 
same  timeliness  problems  as  the  data  reported  in  SARs. 

— An  expanded  requirement  for  DOD  certification  of  the 
validity  of  its  estimates  may  result  in  more  complete 
cost  reporting  to  the  Congress. 
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RECOMMENDATIONS 


To  provide  more  realistic  cost  reports  to  the  Congress,  we 
recommend  that  the  Secretary  ensure  that; 

— SARs  report  all  relevant  program  costs  (such  as  operation 
and  support),  use  the  most  current  data,  and  report  costs 
in  a  consistent  manner.  In  an  exceptional  situation 
where  costs  are  excluded  from  the  estimate,  those  costs 
should  be  clearly  identified  and  the  rationale  for  their 
exclusion  explained. 

— Clear  criteria  are  established  regarding  the  costs  to  be 
included  in  the  officially  approved  program  for  a  weapon 
system. 

— DOD  disclose  the  total  number  of  units  it  is  considering 
for  a  program  by  providing  a  SAR  footnote  when  that  num¬ 
ber  is  different  from  the  approved  program  reported  in 
the  body  of  the  SAR. 

— Unit  cost  exception  reports  disclose  any  anticipated 
cost  growth  that  has  not  been  included  in  the  latest 
officially  approved  estimate. 

MATTERS  FOR  CONGRESSIONAL  CONSIDERATION 

Although  the  Congress  has  deliberated  at  great  length  over 
the  credibility  of  cost  estimates  it  receives  from  DOD,  a  solu¬ 
tion  to  the  problem  still  is  not  at  hand.  The  Congress,  or  its 
oversight  committees,  have  required  DOD  to  certify  as  to  the 
validity  of  cost  estimates  on  particular  systems.  The  B-1B  bom¬ 
ber  program  is  a  recent  example.  Also,  as  a  result  of  legisla¬ 
tion,  unit  cost  exception  reports  are  required  to  be  submitted 
by  DOD  to  the  oversight  committees  when  certain  cost  thresholds 
are  breached.  We  suggest  that  the  Congress  may  want  to  require 
DOD  to  certify  that  the  cost  estimates  it  reports  are  prepared 
according  to  sound  cost  estimating  guidelines,  and  represent  the 
total  cost  for  the  weapon  systems  program.  Programs  in  danger 
of  breaching  these  estimates  should  undergo  a  rigorous 
evaluation  by  DOD  and  the  Congress.  (See  pp.  24  and  25.) 

DOD  COMMENTS 


The  following  summarizes  DOD's  comments  on  this  chapter  and 
provides  our  responses; 

DOD  comment 


DOD  did  not  agree  with  any  of  our  findings  regarding  DOD 
reports  to  the  Congress.  DOD  stated  that  the  improvements  we 
suggested  are  not  possible,  and  addressed  each  one  individually 
as  follows; 
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(1)  DOD  said  the  SARS  can  only  report  the  latest  approved 
DOD  program,  and  that  this  is  necessary  if  DOD  is  to  be 
allowed  to  do  proper  planning  and  evaluation  of  program 
alternatives . 

Our  response 

It  is  reasonable  that  DOD  should  have  the  opportunity 
to  plan  and  evaluate  alternatives  to  their  programs; 
however,  this  should  not  permit  DOD  to  exclude  from  its 
estimates  costs  that  are  recognized  to  be  necessary  to 
complete  the  weapon  system  acquisitions.  Our  examples 
illustrate  that  DOD  has  excluded  costs  from  its  esti¬ 
mates  or  delayed  the  reporting  of  costs  that  would  give 
the  Congress  the  information  it  seems  to  want  and 
needs  to  be  better  informed.  (See  pp.  38  and  39.) 

(2)  DOD  states  that  the  congressional  data  sheets  and  the 
SARs  reflect  the  Secretary  of  Defense  approved  program  con¬ 
tained  in  the  Five-Year  Defense  Program. 

Our  response 

DOD  Instruction  7000.3  requires  DOD  to  report  all 
costs — not  just  the  Five-Year  Defense  Program.  The 
instruction  requires  DOD  to  report  all  program  costs, 
including  current  and  prior  year  costs;  budget  year 
costs;  balance  to  complete  the  Five-Year  Defense  Pro¬ 
gram;  and  the  balance  to  complete  the  program  acquisi¬ 
tion.  Our  report  example  illustrates  this  problem. 

(See  p.  39.) 

(3)  DOD's  comment  indicates  that  DOD  is  reluctant  to 
include  operation  and  support  costs  in  SAR  because  of  the 
lack  of  precision  inherent  in  the  estimating  of  operation 
and  support  costs. 

Our  response 

We  feel  that  estimates,  including  operation  and  support 
costs,  would  be  of  value  to  the  Congress.  No  operation 
and  support  costs  are  reported  now.  Using  the  opera¬ 
tion  and  support  estimates  DOD  currently  develops  would 
fill  this  gap.  (See  p.  40.) 

(4)  DOD  states  that  generally,  SARs  report  costs  consis¬ 
tently  and  that  SAR  policy  permits  the  exclusion  of  costs 
under  certain  circumstances. 
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Our  response 


The  examples  in  our  report  illustrate  that  DOD ' s  SARs 
do  not  always  report  costs  consistently.  DOD's  comment 
indicates  that  this  practice  is  allowable  under  current 
SAR  policy — this  policy  should  be  changed.  (See  p.  40.) 

(5)  DOD  states  its  policy  on  unit  cost  reporting  is  the  same 
as  SARs — the  latest  cost  estimate  of  the  approved  program 
will  be  reported. 

Our  response 

The  Congress  passed  the  Unit  Cost  Report  legislation 
because  the  data  the  Congress  received  through  the  SARs 
was  not  timely.  If  the  Congress  believes  the  SAR  data 
is  not  timely  and  DOD  confirms  that  its  policy  toward 
the  unit  cost  reporting  is  the  same  as  its  policy 
toward  SAR  reporting,  then  the  logical  conclusion  is 
that  Unit  Cost  Reports  have  not  solved  the  problem  of 
timeliness  of  the  data  DOD  reports  to  the  Congress  as 
stated  in  our  report.  (See  pp.  40  and  41.) 

Our  overall  response 

to  this  comment 

DOD's  interpretation  of  rules  and  regulations  allows  it  to 
state  the  improvements  suggested  by  us  are  not  possible.  The 
improvements  are  possible — the  rules  and  regulations  that  govern 
DOD  cost  reporting  to  the  Congress  can  be  changed — and  should 
be.  Congressional  actions  clearly  indicate  that  it  wants 
change. 

DOD  comment 


DOD  did  not  concur  with  the  analyses  contained  in  our  draft 
report  which  showed,  by  restating  DOD's  estimates  for  five  of 
the  seven  weapon  systems  we  reviewed,  that  the  current  weapon 
systems  cost  estimates  could  be  substantially  understated  and 
could  cost  over  $65.8  billion — a  potential  understatement  of  $21 
billion. 

DOD  stated  that  it  is  not  correct  to  say  that  DOD  inten¬ 
tionally  and  substantially  understates  cost  estimates.  DOD  fur¬ 
ther  stated  that  the  estimates  were  not  comparable  and  that  dif¬ 
ferent  year  dollars  were  used  in  the  estimates  for  each  system. 

Our  response 

We  agree  that  the  estimates  are  different  and  also  have 
different  year  dollars.  Our  intent  was  to  show  that  for  each 
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system  there  were  numerous  estimates  and  historically  weapon 
systems  costs  tend  to  grow*  However^  it  is  subjective  as  to 
what  will  eventually  be  in  the  programs  we  reviewed  and  we  have 
therefore  deleted  this  section  from  our  report* 
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CHAPTER  4 


RECENT  POD  ACTIONS  SHOULD 

HELP  IF  IMPLEMENTED 


OSD  and  the  services  recognize  the  need  to  improve  their 
cost  estimating  process,  and  as  a  result,  they  are  continually 
taking  some  steps  to  improve  this  capability.  Currently,  OSD, 
the  Air  Force,  Navy,  and  Army  each  have  efforts  underway  to 
improve  their  cost  estimating  practices.  OSD  efforts  are  part 
of  the  DOD  Acquisition  Improvement  Program.  The  current  efforts 
underway  by  the  Air  Force  and  the  Navy,  for  the  most  part, 
appear  to  be  a  reemphasis  and  reinstitution  of  past  measures. 

On  the  other  hand,  in  1982  the  Army  started  what  appears  to  be 
an  extensive  effort  to  improve  its  cost  estimating  with  several 
new  initiatives. 

OSD  EFFORTS  UNDERWAY  TO 

IMPROVE  COST  ESTIMATING 

In  April  1981  the  Deputy  Secretary  of  Defense  set  out  32 
initiatives  to  be  pursued  to  improve  the  DOD  acquisition  pro¬ 
cess.  A  Task  Force  on  Acquisition  Improvement  was  established 
to  do  a  comprehensive  review  of  the  initiatives.  As  of  June 
1983,  13  of  the  initiatives  were  considered  completed  and  9 
required  some  further  action.  The  others  were  merged  into  six 
major  areas:  (1)  program  stability,  (2)  multiyear  procurement, 
(3)  economic  (stable)  production  rates,  (4)  realistic  budgeting, 
(5)  improved  readiness  and  support,  and  (6)  encouraging  competi¬ 
tion.  The  realistic  budgeting  initiative  will  consider  budget¬ 
ing  to  most  likely  cost,  technological  risk  funding,  and  budget¬ 
ing  for  inflation.  Improvements  in  all  of  these  areas  should 
also  have  an  effect  on  better  cost  estimating  and  reporting. 

EFFORTS  UNDERWAY  BY  THE  AIR  FORCE 

AND  THE  NAVY  TO  IMPROVE  COST  ESTIMATING 


Some  of  the  ways  the  Air  Force  and  the  Navy  are  trying  to 
improve  their  cost  estimating  capabilities  are  through  addition¬ 
al  (1)  cost  estimating  research,  (2)  use  of  computers,  and  (3) 
recruiting  and  training  of  cost  estimators.  In  addition,  the 
Navy  is  considering  additional  cost  estimating  guidance.  The 
Air  Force  appears  to  be  reemphasizing  its  baseline  program 
management  system  with  some  modifications.  This  system  is 
designed  to  be  a  contract  between  the  program  manager  or  project 
office  and  the  Air  Force  Systems  Command  or  field  organization 
commander.  The  contract  should  reflect  the  program  director/ 
manager's  best  estimate  of  the  cost  to  accomplish  the  program 
objectives.  The  system  was  to  establish  and  maintain  cost  dis¬ 
cipline  and  track  costs.  The  objective  of  the  new  effort  is  to 
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involve  more  players  in  the  contract  process  and  to  establish  a 
mutual  understanding  among  all  major  organizations  participating 
in  the  weapon  systems  acquisition  of  what  the  program  will  con¬ 
sist  of  before  formalizing  the  agreement  or  "contract."  The 
contract  is  to  define  the  scope  of  the  program  in  terms  of  sche¬ 
dule,  cost,  and  identification  of  items  not  included  in  the  pro¬ 
gram  objectives.  The  Air  Force  appears  to  be  making  another 
effort  to  implement  this  system  under  "Project  Cost." 

Project  Cost,  initiated  in  early  1983,  is  a  program  which 
has  pulled  together  all  Air  Force  Systems  Command  cost  control 
measures  and  added  some  new  ones.  Over  200  actions  are  being 
undertaken  within  Project  Cost  concentrating  on  three  areas: 

(1)  weapon  system  affordability,  (2)  program  stability,  and  (3) 
program  management.  The  Air  Force  has  selected  10  of  these 
actions  to  receive  special  merit. 

— Stringent  review  of  contractor  wages. 

— A  new  baselining  effort. 

— Reduction  of  data/documentation  required  from  contrac¬ 
tors. 

— Increase  program  emphasis  up  front. 

— Stabilizing  programs. 

— Designing  and  managing  with  available  funding. 

— Increased  effective  competition. 

— Increased  use  of  "should"  costs. 

— Program  manager  accountability. 

— Improvement  in  cost  estimating. 

EFFORTS  TO  IMPROVE  THE 

ARMY'S  COST  ESTIMATING 


The  Army  has  approximately  45  initiatives  planned  or  in 
process  to  improve  cost  discipline  on  its  major  weapon  systems. 
Many  of  the  initiatives  are  in  the  early  stages  of  implementa¬ 
tion  or  in  the  pilot  implementation  stages.  In  some  cases,  only 
interim  instructions  have  been  established  with  the  intention  of 
establishing  formal  regulations  after  determining  if  the  concept 
is  feasible  and  what  improvements  or  changes  are  needed.  The 
initiatives  that  relate  to  cost  estimating  include: 

— Total  Risk  Assessing  Cost  Estimate  for  Production, 

— Program  Management  Control  System, 

— vioint  Independent  Cost  Estimating  teams, 

use  of  major  cost  drivers  to  validate  cost  estimates, 

— annual  updating  of  program  manager's  cost  estimates, 

— Risk  Review  teams, 

— reducing  the  size  of  the  Materiel  Systems  Requirements 
Specifications  which  serves  as  the  basis  for  cost 
estimates , 

— single  best  cost  estimate  from  the  Army  Materiel  Develop¬ 
ment  and  Readiness  Command,  and 
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— displaying  uncertainty  ranges  in  cost  estimates. 

The  Total  Risk  Assessing  Cost  Estimate  for  Production, 
Program  Management  Control  System,  and  Joint  Independent  Cost 
Estimating  team  are  three  of  the  major  initiatives. 

Total  Risk  Assessing  Cost 

Estimate  for  Production 


By  an  October  6,  1982,  Letter  of  Instruction,  the  Army 
initiated  the  concept  of  funding  for  technological  risk  for  the 
early  years  of  production,  known  as  Total  Risk  Assessing  Cost 
Estimate  for  Production.  It  is  to  provide  consistency  in  pro¬ 
curement  planning,  programming,  and  budgeting  under  conditions 
of  risk  and  uncertainty.  The  concept  involves  identifying  and 
quantifying  risks  when  a  weapon  system  transitions  from  develop¬ 
ment  to  production,  submitting  the  risk  cost  with  the  baseline 
estimate,  and  reserving  funds  to  cover  the  expected  cost  of  the 
uncertainty . 

Although  the  Army  is  the  only  service  at  this  time  with 
Total  Risk  Assessing  Cost  Estimate  for  Production,  each  of  the 
services  have  implemented  similar  techniques  for  budgeting  for 
technological  risk  in  the  research  and  development  phase  of  wea¬ 
pon  system  acquisition. 

In  developing  the  Total  Risk  Assessing  Cost  Estimate  for 
Production,  specific  categories  of  risk  have  been  identified 
such  as  threat  uncertainty  and  design  changes.  However,  speci¬ 
fically  excluded  from  Total  Risk  Assessing  Cost  Estimates 
for  Production's  expected  risk  costs  are  factors  such  as  unknown 
unknowns,  inadequate  funding  in  early  years,  and  quantity 
changes . 

Program  Management  Control  System 

The  objectives  of  this  system  are  to  define  program 
objectives,  provide  increased  discipline  in  the  materiel  acqui¬ 
sition  process,  track  program  execution  against  general  elements 
of  approved  acquisition  strategy,  provide  increased  visibility 
of  program  trends,  and  earlier  identification  of  decision  alter¬ 
natives.  In  addition,  it  is  to  define  cost  baselines  with  an 
auditable  track  to  the  budget,  formalize  a  program  change  pro¬ 
cess,  and  require  monthly  status  against  a  formal  program  base¬ 
line  . 

Joint  Independent  Cost  Estimating  team 

Independent  cost  estimates  and  cost  analysis  briefs  are  now 
being  prepared  by  a  team  consisting  of  representatives  from  the 
Comptroller  of  the  Army,  Army  Materiel  Development  and  Readiness 
Command,  and  major  subordinate  commands.  The  team  is  directed 
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to  concentrate  on  significant  costs  in  each  life-cycle  phase, 
undertaking  sensitivity  analyses  for  those  cost  drivers  critical 
to  the  cost  estimate. 

PROBLEMS  THAT  WILL  AFFECT  THE 

SUCCESSFUL  IMPLEMENTATION  0F~ 

RECENT  ARMY  INITIATIVES 

Although  the  Army  initiatives  are  in  the  early  stages  of 
implementation,  problems  have  already  been  observed.  Some  com¬ 
mand  and  program  office  officials  are  not  receptive  to  some 
aspects  of  the  new  cost  estimating  and  reporting  system.  We 
found  resistence  and  delays  to  implementing  the  Total  Risk 
Assessing  Cost  Estimate  for  Production,  the  Program  Management 
Control  System,  and  Joint  Independent  Cost  Estimating  team  ini— 
tiatives.  For  example,  we  found  that  Apache  program  office 
officials  are  reluctant  to  implement  the  Total  Risk  Assessing 
Cost  Estimate  for  Production  concept  because  they  are  concerned 
that  reserves  for  risk  contingencies  will  be  viewed  as  budget 
"fat"  and  will  be  cut  from  their  funding  quickly.  The  Program 
Management  Control  System  implementation  was  delayed  because  of 
disagreements  between  the  program  office  and  Army  Headquarters 
about  which  cost  estimate  to  use  as  the  basis  for  future  analy¬ 
sis  of  cost  growth.  The  Joint  Independent  Cost  Estimating  con¬ 
cept  is  opposed  by  some  Army  Aviation  Research  and  Development 
Command  officials  who  believe  that  cost  estimators  outside  the 
command  will  not  have  the  time,  resources,  or  expertise  to  ade¬ 
quately  evaluate  the  costs  of  the  weapon  system. 

In  addition,  the  Total  Risk  Assessing  Cost  Estimate  for 
Production  excludes  risk  categories  that  are  characteristic 
of  all  major  weapon  system  programs,  and  if  they  are  not  consi¬ 
dered,  then  weapon  system  costs  will  continue  to  be  understated. 

CONCLUSIONS 


In  the  past  2  years,  DOD  and  the  services  have  initiated 
efforts  to  improve  cost  estimating  and  reporting.  DOD's 
Acquisition  Improvement  Program  includes  several  initiatives 
which,  if  implemented,  should  improve  cost  estimating.  These 
initiatives  address  program  budgeting,  inflation  indexes,  and 
program  restructuring.  The  services,  especially  the  Army,  have 
also  begun  to  update  guidance  and  improve  estimating  proce¬ 
dures.  We  believe  that  the  OSD  and  service  initiatives  are 
needed  to  help  ensure  effective  cost  estimating  and  should  be 
implemented  in  conjunction  with  our  recommendations  presented  in 
chapters  2  and  3. 


\ 
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Th :  HoiiOrajle  Charles  k.  Bowsher 
Ca=pcroil£r  General 
General  Accc-uncing  Office 
A41  G  Scree-,  N.W. 

Kashingrci.  D.C.  20543 

Dear  Chuck.: 


As  you  know,  cy  Cocmiccea  has  had  a  strong  interest  in  how 
effectively  and  efficiently  the  Defense  Departnent  t-anages  its 
accuisiticn  process.  We  have  held  several"  days  of  hearings  con- 
ce^.ing  Defense  Departnenc  acquisition  procedures  and  have  been 
•working  to  aunhorice  an  Inspector  General  in  the  Department  as  ’well. 

The  recent  controversy  surrounding  the  Defense  Deoartment ' s 
cost  estimate  for  the  3-lB  bomber  raises  vary  clearly  one _ of  the 
central  problems  affecting  the  Department  of. Defense  acquisition 
process:'  the  ef f ectiveness  of  cost  estimating  procedures  in  the^ 
Department  for  major  weapons  systems.  The  Defense  Depart¬ 
ment,  in  the  3-13  case,  apparent'ly  hed  developed  several  _  cost 
estimates  for  the  bomber  which  reached  different  conclusions  than 
the  estimates  supplied  to  Congress.  As  you  know,  _ I  supported  the 
Gao's  efforts  to  obtain  cost  analyses  and  supporting  data  used 
bv  the  Defense  Department  to  develop  the  cost  estimates  sent  to 
Congress  on  the  plane  and,  with  several  of  my  Cemnitbse  colleagues, 
sent  a.  letter  to  the  Secretary  of  Defense  urging  their  coo?era  =  ic=* 

Cne  of  uhe  reasons  rhis  case  became  sc  ccncencrous  rs 
the  face  char  no  cne  knows  exactly  how  the  ^Defense  Deparmenc^ 
davelcns  its  cose  estimates,  how  the  cost  data  submtted  to  wT.e 
Defense  Department  by  the  contractors  is  used  in  asscssrn| 
orojected  costs  nor  how  the  Department  decides  on  which  or  the^ 
nany  cost  estimates,  developed  internally  and^by  contractors,  rt 
will  select  as. the  official  estioate  and  submit  to  Congress. 


requesting 
to  assess 

process  in 
to  the  sub 
best  appro 
exa.nples  c 
na  • or  actu 


I  believe  this  issue  deserves  a  thorough  revrew.  1  an 
that  the  General  Accounting  Office  conduct  a  study 
the  cost  estinnting  process  in  the  Defense  Dcpartnenc 
acquisitions.  I  would  expect  the  study  to  review  the 
its  entirety  fron  initial  dcvelcpnent  of  cost  estinates 
nission  of  a  final  estimate  to  Congress.  Perhaps  the  ^ 
ach  to  use  for  such  a  study  is  to  examine  several,  nctua.^ 
i'"hcv  the  cost  estimating  process  worked  for  specific 
.i^s'LzLor^s ,  The  study  should  net  exclude  an  exentnatton 


50 


APPEIsDlIv  I 


APPENDIX  I 


Ch/irles  A.  Bowsher 
?r.se  2 


*^v  and  cost  analyses^  and 

Che  Decense  Deparcnenc  in  che  devalopnan 


how  cn:'.y  are  uciliiicd  b 
of  cjsc  escisaces. 


I 

Your  scaff 
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chert  are  any  questions. 
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POD  COST  ESTIMATING  GUIDANCE 

POD  Directive  5000.1/  Major  System  Acquisitions ^  March  29, 
1982 — This  directive  states  DOD's  acquisition  policy  for  major 
systems  or  major  modifications  to  existing  systems.  It  directs 
DOD  components  to  estimate  and  budget  realistically,  and 
adequately  fund  procurement  (research,  development  and 
production),  logistics,  and  manpower  for  major  systems. 

DOD  Instruction  5000.2,  Major  System  Acquisition  Procedures, 
March  8,  1983 — This  instruction  replaced  DOD  Instruction  5000.2 
dated  March  19,  1980  (see  below),  and  revises  procedures  for 
DOD's  implementation  of  DOD  Directive  5000.1. 

DOD  Instruction  5000.2,  Major  System  Acquisition  Procedures, 
March  19,  1980 — This  instruction  provides  the  supplementary  pro¬ 
cedures  for  DOD  use  in  implementing  DOD  Directive  5000.1  (see 
above).  Regarding  estimates,  the  instruction  states: 

"The  validity  of  decisions  reached  at  each  mile¬ 
stone  depends  upon  the  quality  of  cost,  schedule, 
performance,  and  supportability  estimates  pre¬ 
sented  at  the  milestone  reviews.  Although  there 
is  considerable  uncertainty  early  in  the  acquisi¬ 
tion  process,  every  effort  must  be  made  to  use 
the  best  available  data  and  techniques  in  devel¬ 
oping  estimates.  Bands  of  uncertainty  shall  be 
identified  for  point  estimates.  Broad  bands  of 
uncertainty  shall  be  expected  early  in  the  acqui¬ 
sition  process,  with  smaller  bands  developed  as 
the  program  matures  and  uncertainty  decreases. 

Traceability  of  successive  cost  estimates,  to 
include  adjustments  for  inflation  and  to  segre¬ 
gate  estimating  error  from  program  changes,  shall 
be  maintained  starting  with  program  cost  esti¬ 
mates  approved  at  Milestone  I."^ 

DOD  Directive  5000.28,  Design  to  Cost,  May  23,  1975 — This 
directive  establishes  policy,  guidance,  and  responsibilities  of 
Design  to  Cost  principles  to  the  acquisition  of  defense  systems, 
subsystems,  and  components. 

DOD  Instruction  5000.33,  Uniform  Budget/Cost  Terms  and 
Definitions ,  August  15,  1977 — This  instruction  has  a  threefold 
purpose ; 


^Concept  selection  and  entry  into  the  demonstration  and 
validation  phase  of  the  acquisition  of  a  new  system. 
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— establishes  uniform  budget/cost  terms  for  use 
in  the  management  of  DOD  weapon/support  sys¬ 
tems; 

— presents  the  framework  of  cost  terms  and  defi¬ 
nitions  which  include  the  basic  components  of 
cost  associated  with  DOD  weapon/support  sys¬ 
tems;  and 

— identifies  cost  categories,  work  breakdown 
structure  elements,  and  the  appropriations 
associated  with  each  cost  term,  as  the  means  of 
improving  the  completeness,  consistency,  use, 
and  understanding  of  these  terms  in  the  DOD 
decision  process. 

DOD  Directive  5000.4,  OSD  Cost  Analysis  Improvement  Group, 
October  30,  1980 — This  directive  provides  the  permanent  charter 
for  the  OSD  Cost  Analysis  Improvement  Group.  It  specifies  the 
organization,  responsibilities,  and  administration  of  the  Group 
and  includes  the  criteria  and  procedures  for  the  preparation  and 
presentation  of  cost  analyses  to  the  OSD  Cost  Analysis 
Improvement  Group. 

DOD  Directive  5010.20,  Work  Breakdown  Structures  for  Defense 
Materiel  Items,  July  31,  1968 — This  directive  establishes 
DOD  policy  governing  the  preparation  and  application  of  work 
breakdown  structures  for  use  during  the  acquisition  of  systems, 
equipment,  or  other  materiel  items. 

DOD  Instruction  7000.2,  Performance  Measurement  for  Selected 
Acquisitions,  June  10,  1977 — This  instruction  sets  forth  objec¬ 
tives  and  criteria  for  the  application  of  uniform  DOD 
requirements  to  selected  defense  contracts.  The  provisions  of 
this  instruction  specifically  require  the  use  of  Cost/Schedule 
Control  Systems  Criteria  in  selected  acquisitions. 

DOD  Instruction  7000.3,  SARs ,  March  2,  1983 — This 
instruction  provides  standardized  format  and  instructions  to  be 
followed  by  DOD  components  in  responding  to  Secretary  of  Defense 
requirements  for  summary  reporting  of  technical,  schedule, 
quantity,  and  cost  information  concerning  major  defense 
programs.  The  SAR  is  established  as  the  standard,  comprehensive 
summary  status  report  on  major  defense  systems. 

DOD  Instruction  7000.11,  Contractor  Cost  Data  Reporting, 
September  5,  1973 — This  instruction  provides  guidance  for 
collecting  projected  and  actual  cost  data  on  acquisition  programs 
from  contractors  and  in-house  government  plants  through  a  single 
integrated  system  for  DOD  cost  analysis  and  procurement 
management  purposes. 
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POD  Instruction  7041. 3>  Economic  Analysis  and  Program  Evalu¬ 
ation  for  Resource  Management,  October  18,  1972 — This  instruction 
outlines  policy  guidance  and  establishes  a  framework  for 
consistent  application  of 

— economic  analysis  of  proposed  programs,  proj¬ 
ects,  and  activities  and 

— program  evaluation  of  on-going  activities. 

POD  Instruction  7045.7,  The  Planning,  Programming,  and 
Budgeting  System,  October  29 ,  1969 — This  instruction  establishes 
procedural  guidance  for 

— processing  changes  to  the  approved  resources  of 
the  Five-Year  Defense  Program; 

— submission,  analysis,  review,  and  approval  of 
new  and  revised  DOD  programs  and  budgets;  and 

— maintenance  and  updating  of  the  Five-Year  Defense 
Program  structures. 

OSD  Cost  Analysis  Improvement  Group  Aircraft  Operating  and 

Support  Cost  Development  Guide  (Draft) — This  guide  provides 
guidelines  for  preparing  and  presenting  estimates  of  operating 
and  support  cost  to  the  OSD  Cost  Analysis  Improvement  Group  and 
DSARC.  It  also  provides  definitions  for  operating  and  support 
cost  elements  used  in  the  cost  oriented  resource  estimation 
model . 

Sample  Operating  and  Support  Analysis,  Operating  and  Support 

Cost  Estimates  for  Air  Force  FX  Aircraft,  DSARC  III,  prepared  for 
the  Secretary  of  Defense,  Cost  Analysis  Improvement  Group, 

January  1,  1980, — This  sample  is  designed  to  complement  the  Cost 
Analysis  Improvement  Group's  Cost  Development  Guide.  It  is 
intended  to  show  an  example  for  how  operating  and  supporting 
costs  can  be  developed  for  the  Cost  Analysis  Improvement  Group 
review  with  available  data  bases  and  one  example  of  an 
appropriate  format  for  presentation  of  cost  estimates. 

Military  Standard  881  A,  Work  Breakdown  Structures  for 
Defense  Materiel  Items,  April  25,  1975 — This  standard  establishes 
criteria  governing  the  preparation  and  employment  of  work 
breakdown  structures  for  use  during  the  acquisition  of  designated 
defense  materiel  items.  The  standard  specifies  that  the  DOD 
component  project  management  will  use  the  work  breakdown 
structure  as  a  coordinating  medium  in  cost  estimates. 
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Army  Regulation  No.  1-1 f  Planning,  Programming,  and 
Budgeting  within  the  Department  of  the  Army,  May  25,  1976 — The 
purpose  of  this  regulation  is  to  prescribe  the  Army's  Planning, 
Programming,  and  Budgeting  System  and  its  relationship  to  the  DOD 
Planning,  Programming,  and  Budgeting  System  and  Joint  Strategic 
Planning  System. 

Department  of  the  Army  Pamphlet  No.  11-1,  Guide  for  Improved 
Use  of  Defense  Documentation  Center  by  Cost  Analysts, 

January  1976 — The  purpose  of  this  guide  is  to  facilitate  better 
interchange  of  data  among  cost  analysts  and  organizations  through 
improved  use  of  Defense  Document  Center  programs,  products,  and 
services . 

Department  of  the  Army  Pamphlet  No.  11-2,  Research  and 
Development  Cost  Guide  for  Army  Materiel  Systems,  May  1976 — The 
purpose  of  this  guide  is  to  provide  a  framework  for  the 
presenting,  documenting,  and  reporting  of  cost  estimates  of  the 
Research  and  Development  phase  of  a  materiel  system's  life 
cycle.  It  establishes  minimum  standards  for  presenting  and 
documenting  these  cost  estimates. 

Department  of  the  Army  Pamphlet  No.  11-3,  Investment  Cost 
Guide  for  Army  Materiel  Systems,  April  1976 — The  purpose  of  this 
guide  is  to  provide  a  framework  for  preparing,  documenting,  and 
reporting  cost  estimates  of  the  investment  phase  of  a  materiel 
system's  life  cycle.  It  establishes  minimum  standards  for 
presenting  and  documenting  these  cost  estimates. 

Department  of  the  Army  Pamphlet  No.  11-4,  Operating  and 
Support  Cost  Guide  for  Army  Materiel  Systems,  April  1976 — The 
purpose  of  this  guide  is  to  provide  a  framework  for  presenting, 
documenting,  and  reporting  cost  estimates  of  the  operating  and 
support  phase  of  a  materiel  system's  life  cycle.  It  establishes 
minimum  standards  for  presenting  and  documenting  these  cost 
estimates. 

Army  Pamphlet  No.  11-5,  Standards  for  Preparation  and 
Documentation  of  Life-Cycle  Cost  Estimates  for  Army  Materiel 

Systems ,  May  1976 — This  guide  provides  a  framework  for  the 
presentation  and  life-cycle  cost  estimates  of  major  Army  materiel 
systems.  It  states  that  it  was  prepared  for  use  by  Army  cost 
analysts  in  standardizing  the  presentation  and  documentation  of 
life-cycle  cost  estimates. 

Army  Regulation  No.  11-18,  The  Cost  Analysis  Program, 

October  10,  1975 — This  regulation  establishes  Department  of  the 
Army  policy  and  procedures  governing  Army-wide  cost  analysis.  It 
assigns  responsibilities  and  outlines  command  and  staff 
relationships  for  the  cost  analysis  program. 


55 


APPENDIX  II 


APPENDIX  II 


Army  Regulation  No«  15-14,  Systems  Acquisition  Review 
Council  Procedures^  April  1,  1978 — This  regulation  provides 
guidance  and  establishes  procedures  governing  ASARC. 

Army  Regulation  No.  1000-1,  Basic  Policies  for  Systems 
Acquisition,  April  1,  1978 — This  regulation  establishes  the  basic 
Army  policy  for  acquisition  of  materiel  systems,  and  together 
with  Army  Regulation  15-14,  implements  DOD  Directives  5000.1  and 
5000.2. 


U.S.  Army  Materiel  Development  and  Readiness  Command 
Regulation  37-4,  Cost  Estimate  Control  Data  Center  Activities, 
October  4,  1982 — This  regulation  prescribes  the  policies, 
responsibilities,  and  procedures  for  the  operation  of  Cost 
Estimate  Control  Data  Centers  within  Army  Materiel  Development 
and  Readiness  Command.  These  centers  are  to  review  and  validate 
cost  estimates  and  data  before  being  released  for  official  use. 

Letter  of  Instruction  for  Implementation  of  the  Total  Risk 

Assessing  Cost  Estimate  for  Production,  October  6,  1982 — This 
letter  of  instruction  establishes  interim  policy,  procedures,  and 
responsibilities  for  implementing  Total  Risk  Assessing  Cost 
Estimate  for  Production. 

DARCOM  Circular  No.  11-1,  Army  Materiel  Development  and 
Readiness  Command  Program  Management  Control  System, 

September  20,  1982 — This  circular  directs  the  implementation  of 
the  Program  Management  Control  System  with  its  effectiveness  to 
be  evaluated  and  once  proven  to  be  integrated  into  appropriate 
regulation.  It  specifies  the  policy,  procedures,  and 
responsibilities  for  Program  Management  Control  System 
implementation. 

Memorandum  of  Understanding  between  the  Comptroller  of  the 

Army  and  the  Army  Materiel  Development  &  Readiness  Command, 

May  27,  1982,  on  Joint  Independent  Cost  Estimating  teams — This 
memorandum  outlines  procedures  for  establishing  and  operating  a 
joint  team  to  prepare  independent  cost  estimates  for  major  weapon 
systems . 

Letter  on  Annual  Update  of  Baseline  Cost  Estimates  from  the 

Adjutant  General,  September  21,  1981 — This  letter  establishes 

Army  policy  for  baseline  cost  estimates  to  be  updated  at  least 
annually  by  the  program  manager  to  reflect  changes  as  the  systems 
progress  through  the  acquisition  phases. 

Army  Development  and  Readiness  Command  Pamphlet  P700-6, 

Naval  Material  Command  Pamphlet  NAVMAT  P5242,  Air  Force  Logistics 
Command  Pamphlet  AFLCP/AFSCP  800-19,  October  15,  1977,  Joint 
Design- to-Cost  Guide,  Life-Cycle  Cost  as  a  Design  Parameter — 

This  guide  provides  information  and  guidance  for  application  of 
the  design- to-cost  concepts. 
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SECNAV  Instruction  4000.31 r  Life-Cycle  Costing^  December  7, 
1970 — This  instruction  establishes  policy  and  responsibilities 
for  the  development  and  implementation  of  Navy  life-cycle  costing 
policy  and  procedures  for  less  than  major  acquisitions. 

SECNAV  Instruction  5000. lA^  System  Acquisition  in  the 
Department  of  the  Navy,  November  17,  1978 — This  instruction 
promulgates  the  policies  for  weapon  system  acquisition 
established  by  DOD  Directives  5000.1  and  5000.2,  and  provides 
supplemental  acquisition  policy  guidance  within  the  Department  of 
the  Navy. 

SECNAV  Instruction  5420. 172B,  Establishment  of  the 
Department  of  the  Navy  Systems  Acquisition  Review  Council, 

May  18,  1976 — This  instruction  defines  the  organization,  mission, 
functions,  and  procedures  of  the  Navy  Systems  Acquisition  Review 
Council . 

SECNAV  Instruction  7000. 14B,  Economic  Analysis  and  Program 
Evaluation  for  Navy  Resource  Management,  June  18,  1975 — This 
instruction  provides  policy  guidance  and  responsibilities  for 
economic  analysis  and  program  evaluations  within  the  Navy. 

SECNAV  Instruction  7000. 17B,  Contractor  Cost  Performance 
Measurement  for  Selected  Acquisitions,  April  14,  1978 — This 
instruction  prescribes  policy,  objectives,  and  responsibilities 
for  the  application  of  the  Cost/Schedule  Control  Systems  Criteria 
for  selected  Navy  acquisitions. 

SECNAV  Instruction  7000. 19B,  Department  of  the  Navy  Cost 
Analysis  Program,  March  12,  1975 — This  instruction  establishes 
policy  on  cost  estimating  throughout  the  Department  of  the  Navy 
and  assigns  responsibilities  for  estimating,  validating,  and 
reviewing  in  the  cost  analysis  program. 

SECNAV  Instruction  7700. 5D,  Selected  Acquisition  Reports, 
December  19,  1979 — This  instruction  implements  DOD ' s  SAR 
requirements  and  provides  additional  guidance  for  the  staffing 
and  submission  of  these  reports. 

NAVAIR  Instruction  4265.1  A,  Pricing  Information  Associated 
with  Planning,  Programming,  and  Budgeting  of  Aircraft  and  Missile 

Programs,  February  1,  1971 — This  instruction  establishes  the 

responsibilities  and  procedures  for  requesting,  preparing,  and 
disseminating  pricing  information  associated  with  Naval  Air 
Systems  Command  Headquarters  aircraft  and  missile  programs. 

NAVMAT  Instruction  7000. 17D,  Contractor  Cost  and  Schedule 
Performance  Measurement  and  Reporting  for  Major  Acquisition 

Projects,  March  4,  1981 — This  instruction  establishes  the 
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contractor  cost  and  schedule  performance  measurement  and 
reporting  requirements  for  Naval  Material  Command  major  defense 
acquisitions. 

NAVMAT  Instruction  7000. 19A,  Naval  Material  Command  Cost 
Analysis/Estimating  Program,  July  30,  1976 — This  instruction 
implements  the  Department  of  the  Navy's  cost  analysis  program 
within  the  Naval  Material  Command.  It  emphasizes  the  importance 
of  providing  realistic  cost  estimates  in  support  of  the  Navy's 
acquisition  process  and  planning,  programming,  and  budgeting 
efforts.  It  states  that  each  systems  command  is  to  have  a  cost 
analysis/estimating  group  responsible  for  cost  estimating  policy 
and  guidance. 

NAVMAT  Instruction  4330.37,  Should  Cost,  March  25,  1974 — 
This  instruction  provides  guidance  and  procedures  on  the  use  of 
should-cost  studies  in  determining  pricing  of  Navy  contracts. 

NAVSEA  Instruction  7000.9,  Financial  Management  Manual 
Volume  5;  Cost  Estimating  and  Analysis,  July  14,  1980 — This 
instruction  provides  guidance,  document  requirements,  and 
repons ibil i ties  of  NAVSEA  cost  estimating  process. 

NAVSEA  Instruction  7300.10,  Classification  of  Cost 
Estimates ,  January  27,  1977 — This  instruction  provides  policy  and 
guidance  for  a  classification  system  involving  cost  estimates. 

NAVELEX  Instruction  7720. 4A,  Policy  and  Responsibilities  for 
NAVELEX  Cost  Estimating  and  Analysis,  September  9,  1975 — This 
instruction  establishes  a  Central  Cost  Estimating  and  Analysis 
Group  in  NAVELEX  which  will  set  forth  policy,  procedures,  and 
assign  responsibilities  for  the  development  of  system/equipment 
cost  estimates  and  for  performance  of  technical  cost  analyses. 

The  objective  of  the  instruction  is  to  improve  the  credibility 
and  validity  of  cost  estimating  and  analysis  within  the  Naval 
Electronic  Systems  Command. 

OPNAV  Instruction  7000. 17A,  Cost  Analysis,  September  15, 

1976 — This  instruction  provides  supplemental  policy  guidance  for 
further  implementation  of  the  Department  of  the  Navy  Cost 
Analysis  Program. 

OPNAV  Instruction  7000.18,  Economic  Analysis  and  Program 
Evaluation  for  Navy  Resource  Management,  July  27,  1973 — This 
instruction  provides  policy  guidance  and  responsibilities  for 
economic  analysis  and  program  evaluation  under  the  command  of  the 
Chief  of  Naval  Operations. 

OPNAV  Instruction  7000. 19A.  Review  of  Comptroller 
Organizations,  January  24,  1978 — This  instruction  provides 
guidance,  responsibilities,  and  objectives  for  the  review  and 
approval  of  Navy  comptroller  organizations. 
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OPNAV  Instruction  7710. IB,  Ship  Cost  Adjustment  Report^ 

June  19,  1973 — This  instruction  provides  guidance,  procedures, 
and  responsibilities  for  preparing  and  approving  the  Ship  Cost 
Adjustment  report  within  the  Department  of  the  Navy. 

AIR  FORCE  COST  ESTIMATING  GUIDANCE 

Air  Force  Regulation  173—1,  The  Air  Force  Cost  Analysis 
Program,  October  10,  1975 — This  regulation  establishes  the  Air 
Force  Cost  Analysis  Program,  specifies  its  objectives  and 
functions,  and  assigns  responsibilities  for  the  conduct  of  the 
program  within  the  Department  of  the  Air  Force. 

Air  Force  Regulation  173—11,  Independent  Cost  Analysis 
Program,  December  12,  1980 — This  regulation  establishes  the 
Independent  Cost  Analysis  Program,  prescribes  policies,  assigns 
responsibilities,  and  defines  procedures  for  preparing, 
reviewing,  documenting,  and  presenting  studies  done  as  part  of 
the  Independent  Cost  Analysis  Program.  It  outlines  Air  Force 
Cost  Analysis  Improvement  Group  support  provided  to  the  Air  Force 
Systems  Acquisition  Review  Council  and  the  DSARC. 

Air  Force  Regulation  800-5,  Selected  Acquisition  Reports, 
June  6,  1980 — This  regulation  establishes  responsibilities  for 
the  preparation  and  submission  of  SARs. 

Air  Force  Regulation  800-6,  Program  Control-Financial, 
September  7,  1976 — This  regulation  provides  policy,  establishes 
reporting  requirements,  assigns  responsibilities,  and  establishes 
procedures  for  applying  financial  management  control  techniques 
to  Air  Force  acquisition  and  modification  programs.  This 
regulation  is  being  revised. 

Air  Force  Systems  Command  Manual  173-1,  Cost  Estimating 
Procedures,  April  17,  1972 — This  manual  serves  as  a  comprehensive 
reference  for  Air  Force  Systems  Command  cost  estimating  in 
support  of  system  acquisition  activities.  Cost  estimating 
methods,  documentation,  and  review  are  described.  The  techniques 
apply  to  all  persons  who  must  prepare  or  use  cost  estimates. 

Air  Force  Systems  Command  Regulation  173-2,  Confidence 
Levels  of  Cost  Estimates,  October  2,  1978— This  regulation 
prescribes  policy,  responsibilities,  and  procedures  for  rating 
cost  estimates  prepared  according  to  Air  Force  Systems  Command 
Manual  173-1  and  Air  Force  Regulations  173-1  and  173-11.  This 
regulation  provides  a  system  to  rate  the  confidence  levels  of 
cost  estimates  considering  the  methods  used  in  preparing  the 
estimates  and  the  availability  of  data. 
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Air  Force  Systems  Command  Regulation  550-182  Program 
Baseline  Management,  October  20,  1980 — This  regulation  describes 
the  policy  and  procedures  for  Program  Baseline  Management  within 
the  Air  Force  Systems  Command.  It  states  that 

"The  Program  Baseline  is  a  "contract"  between  the 
program  or  project  office  (PO)  and  the  AFSC  Commander 
or  field  organization  Commander  which  reflects  the 
Program  Director/Manager's  best  estimate  of  the  cost 
to  accomplish  the  effort  described  in  current 
direction.  This  contract  accomplishes  two  main 
objectives;  (1)  to  establish  and  maintain  program 
cost  discipline  and  (2)  to  provide  an  unequivocal 
program  cost  track.  The  Baseline  is  independent  of 
program  funding." 

Air  Force  Systems  Command  Regulation  800—6,  Program  Baseline 
Management ,  December  3,  1981 — This  regulation  describes  the 
policy  and  procedures  for  program  baseline  management  within 
the  Air  Force  Systems  Command.  It  applies  to  all  acquisition 
programs  at  Air  Force  Systems  Command  product  divisions  and 
centers  and  to  selected  Headquarters  Air  Force  System  Command 
managed  programs  at  the  laboratories.  The  program  baseline  is  a 
"contract"  between  the  program  or  project  manager  and  the 
designated  baseline  approval  authority  that  reflects  the  program 
manager's  best  estimate  of  the  cost  to  accomplish  the  effort 
described  in  current  direction.  The  baseline  will  not 
necessarily  equate  to  current  approved  funding.  The  contract  has 
two  main  objectives; 

(1)  Sets  and  keeps  program  cost  discipline. 

(2)  Provides  an  unequivocal  program  cost  track  that  is 
documented  in  a  standardized  format. 

Air  Force  Systems  Command  Pamphlet  173-3,  Cost/Schedule 
Management  of  Nonmajor  Contracts  (Joint  Guide),  November  1, 

1978 — This  pamphlet  provides  procedures  for  the  cost/schedule 
management  of  nonmajor  contracts  and  in  particular,  the 
implementation  and  use  of  the  Cost/Schedule  Status  Report. 

Air  Force  Systems  Command  Pamphlet  173-5,  Cost/Schedule 
Control  Systems  Criteria  Joint  Implementation  Guide,  October  1 , 
1980  This  pamplet  provides  procedures  for  use  during  planning 
and  implementation  of  Cost/Schedule  Control  Systems  and  for 
surveillance  of  contractor  compliance. 


^This  regulation  has  been  incorporated  into  Air  Force  Systems 
Command  Regulation  800-6. 
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Air  Force  Logistics  Command  and  Air  Force  Systems  Command 

Pamphlet  173-6,  Cost/Schedule  Control  Systems  Criteria  Joint 
Surveillance  Guide,  July  1,  1974 — This  pamphlet  provides  uniform 
guidance  for  the  Military  Departments  and  other  Defense  agencies 
responsible  for  surveillance  of  Cost/Schedule  Control  Systems 
Criteria  consistent  with  the  provisions  of  DOD  Instructions 
7000.2. 

Air  Force  Systems  Command  Pamphlet  800-15,  Contractor  Cost 
Data  Reporting,  November  5,  1973 — This  pamphlet  provides  uniform 
procedures  which  have  been  approved  for  use  in  implementing  and 
administering  the  Contractor  Cost  Data  Reporting  System. 

Air  Force  Logistics  Command  Pamphlet  173—3,  A  Guide  for 
Estimating  Aircraft  Logistics  Support  Costs,  March  12,  1974 — 

This  pamphlet  provides  guidance  for  estimating  logistics  costs 
for  new  systems  and  addresses  the  Air  Force  Aircraft  Procurement 
Appropriation  (3010)  and  the  depot  maintenance  portion  of  the 
Operations  and  Maintenance  Appropriation  (3400). 

Aeronautical  Systems  Division  Regulation  173-1,  Aeronautical 
Systems  Division  Cost  Analysis  Program,  October  21,  1981 — This 
regulation  establishes  policy,  operation  procedures,  fundamental 
concepts,  and  responsibilities  of  the  Comptroller  Staff,  Program 
Offices,  Comptroller  Senior  Collocates,  and  other  organizations 
within  Aeronautical  Systems  Division  that  perform  cost  analysis, 
develop  cost  estimates,  and  conduct  cost  studies.  The 
Aeronautical  Systems  Division  Cost  Analysis  Program  has  been 
established  to  support  the  purpose  and  objectives  of  Air  Force 
Regulation  173-1,  the  Air  Force  Cost  Analysis  Program. 

Air  Force  Test  &  Evaluation  Center  Regulation  55-1  ,  Air 
Force  Test  and  Evaluation  Center  Operations  Regulation, 

August  15,  1982 — This  regulation  defines  how  the  Air  Force  Test 
and  Evaluation  Center  will  implement  the  operational  test  and 
evaluation  of  weapon  systems.  It  includes  direction  regarding 
the  initial  funding  estimate. 

Air  Force  Flight  Test  Center  Regulation  170-2, 
Product/Service  Unit,  July  18,  1980 — This  regulation  describes 
methodology,  responsibilities,  establishment  and  validation 
procedures,  and  the  use  and  control  of  product/service  units.  It 
applies  to  all  Air  Force  Flight  Test  Center  organizations  that 
use  product/service  units  for  estimating  and  tracking  costs  and 
resource  use. 
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DESCRIPTIONS  OF  THE  ARMY,  AIR  FORCE,  AND 
NAVY  COST  ESTIMATING  PROCESSES 
ARMY  COST  ESTIMATING  PROCESS 

The  Army's  cost  estimating  process  is  described  in  various 
Army  and  OSD  regulations.  (See  app.  II.)  According  to  these 
regulations — particularly  Army  Regulations  11-18  and  15-4  and 
Army  Materiel  Development  and  Readiness  Command  Regulation 
37—4  cost  estimates  of  major  systems  at  milestone  decisions 
take  the  following  track  and  involve  the  following 
organizations . 

The  materiel  developer  (usually  the  program  manager)  is 
responsible  for  preparing  the  program  cost  estimate. 

— The  cost  estimate  control  data  center,  within  the  cost 
analysis  activity  of  the  applicable  major  subordinate 
command,  performs  a  validation  of  the  program  office  cost 
estimate.  (Army  Materiel  Development  and  Readiness 
Command  Regulation  37-4  describes  validations,  which  is  a 
test  to  confirm  the  cost  estimate.) 

"The  Comptroller  of  the  Army  Cost  Analysis  Division 
is  responsible  for  establishing  a  joint  team  of  the 
Comptroller  of  the  Army,  Development  and  Readiness 
Command,  and  major  subordinate  command  representatives  to 
prepare  an  independent  cost  estimate  and  a  cost  analysis 
brief.  The  cost  analysis  brief  compares  the  program 
office  and  independent  estimates  and  recommends  the 
preferred  Army  estimate. 

— ASARC  recommends  an  Army  cost  position  for  the  Secretary 
of  the  Army's  decision  and  subsequent  recommendation  to 
the  Secretary  of  Defense. 

— The  OSD  Cost  Analysis  Improvement  Group  reviews  and 
evaluates  the  program  office  and  independent  cost 
estimates  and  advises  DSARC  on  cost  estimates. 

— DSARC  recommends  a  cost  position  for  the  Secretary  of 
Defense's  decision. 

These  regulations  also  describe  the  following  estimating 
track  for  annual  cost  estimates  of  major  systems; 

— The  materiel  developer  is  responsible  for  preparing  the 
program  cost  estimate. 

— If  the  annual  cost  estimate  has  only  minor  changes,  the 
cost  estimate  control  data  center  performs  a  validation 
of  the  estimate. 
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— If  the  annual  cost  estimate  has  significant  changes,  the 
cost  estimate  control  data  center  performs  a  level  I 
validation.  Also,  a  joint  team  of  the  Comptroller  of  the 
Army,  Army  Materiel  and  Readiness  Command,  and  major 
subordinate  command  representatives  prepare  an 
independent  cost  estimate,  a  cost  analysis  brief,  and 
arrive  at  an  Army  cost  position. 

Documentation  of  Army  Cost  estimates 

Army  Pamphlets  11-2  through  11-5  specify  that 
"completeness"  (communicating  results  in  a  format  that 
encompasses  the  whole  system)  and  "reproducibility"  (recording 
what  was  done  so  others  may  understand  the  ground  rules, 
assumptions,  analysis,  and  results)  are  the  principles  driving 
cost  estimate  documentation.  According  to  the  Army  pamphlets, 
each  estimate  will  be  a  self-contained  documented  record  and  a 
building  block  for  future  estimates. 

The  pamphlets  also  indicate  that  documentation  is  organized 
on  a  module  basis  so  that  each  cost  element  stands  as  a 
self-contained  unit.  The  basic  building  block  for  documenting 
the  estimate  is  called  the  documentation  module,  consisting  of  a 
cost  data  sheet  and  variable  explanation  sheets  for  each  cost 
expression. 

Review  of  Army  cost  estimates 

According  to  the  Army  Materiel  Development  and  Readiness 
Command  Regulation  37-4,  Army  Regulation  11-18,  and  DOD 
Directive  5000.4,  the  following  reviews  are  required  of  cost 
estimates  for  each  milestone  decision, 

—  independent  cost  estimate, 

— cost  analysis  brief,  and 

— OSD  Cost  Analysis  Improvement  Group  review. 

Army  Regulation  11-18  states  that  an  independent  estimate 
is  developed  to  test  the  reasonableness  of  the  program  office 
cost  estimate  and  to  provide  an  unbiased  second  opinion  of  a 
system's  cost.  The  regulation  describes  the  cost  analysis 
brief  as  a  comparative  analysis  between  the  program  office  and 
independent  estimates  to  surface  and  explain  major  cost 
differences  and  recommend  the  preferred  Army  estimate.  DOD 
Directive  5000.4  describes  the  OSD  Cost  Analysis  Improvement 
Group's  review  as  providing  DSARC  with  a  review  and  evaluation 
of  the  program  office  and  independent  estimates  with  the  purpose 
of  advising  DSARC  on  cost  matters. 
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AIR  FORCE  COST  ESTIMATING  PROCESS 


According  to  Air  Force  Regulation  173-11  and  Areonautical 
Systems  Division  Regulation  173-1,  for  a  major  program 
milestone,  such  as  initiation  of  full-scale  development,  the 
Systems  Program  Office  prepares  a  cost  estimate  and  an 
independent  cost  analysis  that  is  performed  by  the  systems 
division  Directorate  of  Cost  Analysis.  Both  the  Systems  Program 
Office  estimate  and  the  related  independent  cost  analysis  are 
reviewed  by  the  systems  division  Comptroller  and  Commander.  The 
two  estimates  are  then  presented  in  a  series  of  joint  briefings 
by  the  two  teams  through  the  chain  of  command.  The  estimate 
briefings  are  given  to  Air  '^orce  Logistics  Command 
(Comptroller),  Air  Force  stems  Command  (Program  Evaluation 
Group,  Comptroller,  and  Commander),  Headquarters,  U.S.  Air  Force 
(Air  Force  Council,  Comptroller,  Financial  Management,  Cost 
Analysis  Improvement  Group,  and  Air  Force  Systems  Command 
Acquisition  Review  Council!),  and  OSD  (Cost  Analysis 
Improvement  Group^ ) .  Also,  briefings  may  be  made  to  the  using 
commands  for  the  system.  The  briefings  at  each  level  are 
frequently  on  a  joint  basis,  that  is  the  Air  Force  Systems 
Command  briefing  may  be  given  to  all  the  organizations  at  the 
same  time.  For  the  new  Defense  Resource  Board  directed  budget 
independent  cost  analyses,  the  process  is  the  same  except  the 
chain  stops  at  the  OSD  Cost  Analysis  Improvement  Group. 

The  System  Program  Office's  budget  estimate  follows 
essentially  the  same  track,  except  no  Air  Force  Systems 
Acquisition  Review  Council  or  DSARC,  up  to  the  Secretary  of  the 
Air  Force.  The  Secretary  of  Defense  reviews  the  Air  Force 
budget  submittal  as  part  of  the  budget  process,  then  provides 
the  DOD  budget  to  the  President  through  0MB. 

Documentation  of  Air  Force  cost  estimates 


As  stated  in  Air  Force  Systems  Command  Manual  173-1; 

"Cost  estimate  documentation  is  a  detailed  record  of  the 
procedures,  data,  environment,  and  events  resulting  in  a 
cost  estimate.  These  records  describe  configuration, 
development,  and  production  schedules,  defined  quantities, 
conditions  related  to  technological  requirements, 
deployment  concepts,  and  operating  plans.  All  known  or 
anticipated  influences  on  the  cost  to  the  government  for 
acquiring  and  operating  a  system  over  a  prescribed  span  of 
time  are  considered  in  estimating;  therefore,  they  should 
be  considered  in  documenting  or  recording  the  cost 
estimating  effort.  Completeness  of  the  cost  estimate  is  an 
important  objective  that  requires  documenting  excluded  as 
well  as  included  costs.  A  detailed  record  is  required  for 
follow-on  cost  estimates,  cost  tracking,  formal  submissions 


^Air  Force  Systems  Acquisition  Review  Council  or  DSARC 
depending  on  which  approval  level  is  required. 
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of  abstracts  of  cost  estimates,  periodic  reviews,  and  other 

requirements . " 

The  manual  further  states  that  due  to  the  variation  in  program 
phases  that  involve  cost  estimate  preparation,  a  similar  wide 
variation  can  be  expected  in  estimate  documentation.  The  manual 
adds,  however,  that  each  estimate  should  conform  to  a  basic 
pattern  and  include  the  following; 

— System  data,  is  a  record  of  descriptive  information  of 
the  system  being  estimated,  including  (1)  the  purpose  of 
the  estimate,  (2)  background  and  scope,  (3)  ground  rules, 
constraints,  and  assumptions,  and  (4)  system  description 
and  areas  of  cost  uncertainty. 

— System  element  data  contains  the  complete  record  of  each 
work  breakdown  structure  item,  including  (1)  item 
description,  (2)  data  sources,  (3)  basic  data,  and  (4) 
estimate  derivation. 

— Supporting  data  covers  the  items  of  additional  backup; 
for  example,  method  of  evaluation.  The  types  of 
supporting  data  and  circumstances  under  which  each  is 
used  are  cost  quantity  curves  and  estimating  relationship 
records . 

— The  system  estimate  should  portray  the  estimating 
structure  and  be  time  phased  as  required. 

— Estimate  confidence,  should  include  a  short  but 
comprehensive  narrative  depicting  the  degree  of 
confidence  in  the  data,  methods,  and  result. 

Further,  Aeronautical  Systems  Division  Regulation  173-1 
one  of  the  product/developer  divisions  under  Air  Force  Systems 
Command — requires  the  System  Program  Office  to  formally  document 
its  annual  estimates  and  sufficiency  reviews  according  to 
criteria  contained  in  the  regulation.  Air  Force  Regulation 
'I73_'11  establishes  further  documentation  requirements  for  both 
the  System  Program  Office  cost  estimates  and  the  independent 
cost  analysis.  According  to  Air  Force  Regulation  173-11, 
documentation  by  both  should  be  complete  enough  to  allow 
reconstruction  of  the  estimate  by  an  independent  party  and 
should  at  least  include; 

— Summary  program  description. 

— Program  ground  rules  (schedule,  quantity,  etc.). 

--Team  membership. 

— Estimating  assumptions. 

— Program  technical  and  physical  parameters  used  in 
generating  the  estimate. 

— Cost  models  or  cost  estimating  relationships 
used  and  their  data  bases  or  sources. 
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— Cost  factors  and  their  derivation  and  data  sources. 

— Cost  improvement  (learning)  curves. 

— Cost  sensitivity  and  risk  analysis. 

Proper  detailed  and  summary  level  tables  of  the  cost 
estimates . 

Cost  tract  ( independent  cost  analysis  only) . 

— Costs  by  fiscal  year. 

Current  program  funding  by  fiscal  year. 

Program  integrated  master  schedule. 

Review  of  Air  Force  cost  estimates 

The  Systems  Program  Office  milestone  estimate  and  the 
related  independent  cost  analysis  are  reviewed  through  the  Air 
Force  chain  of  command,  the  Air  Force  and  OSD  Cost  Analysis 
Improvement  Groups,  and  the  Air  Force  Systems  Acquisition  Review 
Council  or  DSARC.  The  System  Program  Office  annual  budget 
estimates  are  reviewed  through  the  Air  Force  chain  of  command 
and  by  OSD,  and  are  then  provided  to  the  President  by  the 
Secretary  of  Defense  through  0MB. 

THE  NAVY  COST  ESTIMATING  PROCESS 

The  process  of  developing  an  official  Navy  cost  estimate 
begins  in  the  project  offices  within  the  major  systems  commands, 
such  as  Naval  Air  Systems  Command  and  the  Naval  Sea  Systems 
Command.  Although  the  program  manager  is  responsible  for 
developing  program  cost  estimates,  the  cost  estimate  is  actually 
prepared  by  cost  estimating  groups  within  each  of  the  commands 
under  the  Navy  matrix  approach  to  program  management.  The  cost 
estimating  divisions  are  responsible  for  preparing  timely  and 
accurate  cost  estimates  for  DSARC/Def ense  Naval  Systems 
Acquisition  Review  Council  reviews,  budget  submittals,  support 
of  contract  negotiations,  and  design  to  cost  goals. 

According  to  Secretary  of  the  Navy  Instruction  7000. 17A  and 
Chief  of  Naval  Operations  Instructions  7000. 17A  and  7000. 19B, 
once  the  estimate  is  approved  by  the  appropriate  command,  it  is 
generally  reviewed  by  the  following  Navy  organizations; 

Naval  Material  Command — oversees  the  organizaton  and 
operation  of  cost  estimating  groups  within  the  commands 
and  ensures  that  they  have  the  capability  to  produce 
quality  estimates,  and  has  cognizance  over  all  elements 
of  program  acquisition.  According  to  Chief  of  Naval 
Operations  Instruction  7000. 17A,  cost  estimates  should 
include  development,  investment,  and  operating  and 
support  costs  which  conform  to  Military  Standard  881  or 
another  approved  work  breakdown  structure.  Naval 
Material  Command  is  responsible  for  ensuring  the 
maintenance  of  a  cost  data  base  for  major  systems  to 
establish  and  implement  cost  tracking  procedures,  provide 
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a  basis  for  continuous  review  and  evaluation  of 
estimating  performance,  and  require  that  cost  estimates 
be  properly  documented. 

— Navy  Comptroller  Office — ensures  that  SARs  and  other 
congressional  reporting  documents  are  consistent  with 
documented  estimates  for  major  systems  and  fiscal 
limitations  of  the  budget.  It  also  monitors  cost 
analysis  techniques  employed  by  the  Department  of  the 
Navy. 

— Chief  of  Naval  Operations — maintains  independent  cost 
estimating  groups,  ensures  that  realistic  estimates  are 
provided  for  Navy  systems,  and  develops  and  maintains 
documenting  and  cost  tracking  procedures  in  addition  to  a 
cost  data  base  for  initial  and  follow-on  cost 
estimating.  According  to  Chief  of  Naval  Operations 
Instruction  7000. 19B,  the  Chief  of  Naval  Operations  is 
also  responsible  for  providing  guidance  and  developing 
estimating  methodologies  that  ensures  that  the  total 
costs  of  acquisition  and  ownership  are  reported  to  Navy 
decisionmakers. 

— Secretary  of  the  Navy — The  Chief  of  Naval  Operations 
provides  program  recommendations  to  the  Secretary  for 
approval  in  a  Decision  Coordinating  Paper  which  is 
regularly  updated.  The  Secretary  of  the  Navy  receives 
advisory  support  from  the  Navy  Systems  Acquisition  Review 
Council.  The  Secretary  is  the  highest  decision  authority 
in  the  Navy  and  gives  the  final  approval  for  program 
estimates  designated  for  review.  Other  programs  may  be 
delegated  to  the  Chief  of  Naval  Operations  or  the  Chief 
of  Naval  Material  Command  for  program  approval,  in  which 
case  they  would  be  the  final  authority  for  approval  of 
Navy  cost  estimates. 

Documentation  of  Navy  estimates 

Principal  forms  of  documentation  for  Navy  cost  estimates 
are  the  Decision  Coordinating  Papers,  Cost  Estimate 
Documentation  Summaries  (Naval  Material  Command  Form  7000/2), 
and  budget  documents  specified  in  the  DOD  Budget  Guidance 
Manual  7110-1-M.  The  Navy  regulations  heavily  emphasize  the 
importance  of  documenting,  however,  the  only  specific  document 
format  explained  in  detail  is  the  Cost  Estimate  Documentation 
Summary. 

According  to  Chief  of  Naval  Operations  Instruction 
7000. 17A,  the  Cost  Estimate  Documentation  Summary  will  include 
weapon  system  characteristics,  the  procurement  quantity,  the 
assumed  production  environment,  and  the  resultant  estimated 
costs  (research  and  development,  investment,  and  operating  and 
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support)  to  the  extent  that  they  have  been  estimated  at  that 
point  in  time,  no  matter  how  tentative.  The  instructions 
require  the  form  to  be  color  coded  to  indicate  the  quality  and 
the  reliability  of  the  estimate. 

Chief  of  Naval  Operations  Instruction  7000. 17A  requires  the 
following  elements  of  information  to  be  included  in  documenting 
an  estimate: 

— Date  the  estimate  was  prepared  or  changed. 

— Category  of  the  estimate. 

— Name  of  the  organization  preparing  the  estimate  or 
change . 

— Purpose  of  the  estimate  or  change. 

— Summary  of  characteristics  considered. 

— Ground  rules,  assumptions,  and  constraints. 

— Data  sources. 

— Estimated  cost  and  method  of  derivation. 

— Changes  required  to  the  Resource  Annex  of  the 
Decision  Coordinating  Paper. 

— "Design  to  cost"  requirements. 

— Range  of  uncertainty. 

All  estimates  will  include  a  statement  regarding  the 
confidence  the  estimator  has  in  the  results.  Confidence  could 
be  expressed  in  terras  of  a  range  of  costs.  While  a  detailed 
statistical  treatment  is  desirable,  a  subjective  estimate  of 
the  cost  range  may  be  submitted  with  appropriate  rationale  if 
the  uncertainty  is  not  easily  quantified. 

Chief  of  Naval  Operations  Instruction  7000. 17A  also  states 
that  every  program  manager  will  develop  a  cost  profile  track 
that  would  trace  the  history  of  the  costs  involved  in  the 
program  from  concept  formulation  to  the  present  time.  This  cost 
profile  track  will  graphically  display  changes  in  funding  levels 
with  the  appropriate  rationale  for  such  changes.  It  will  be 
presented  in  the  form  of  a  variance  analysis  that  reflects  all 
Navy  and  DOD  decisions  relative  to  the  program,  as  well  as 
quantity  changes,  engineering  changes,  schedule  changes, 
escalation  changes,  and  so  forth.  This  track,  in  addition  to 
being  available  for  DSARC/Def ense  Naval  Systems  Acquisition 
Review  Council  presentations,  is  updated  and  submitted  as  backup 
information  for  Navy  correspondence  from  Naval  Material  Command 
to  Chief  of  Naval  Operations  relating  to  changes  in  program 
costs. 

Review  of  Navy  estimates 

An  independent  review  of  Navy  estimates  is  primarily  the 
responsibility  of  OP-96D-Secretary  of  the  Navy/Chief  of  Naval 
Operations  Advisor  for  Resource  Analysis  within  the  Office  of 
the  Chief  of  Naval  Operations.  OP-96D  provides  an  independent 
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estimate  of  life-cycle  costs  of  major  weapon  systems  during 
major  milestone  reviews.  This  independent  estimating  group 
provides  a  critical  review  and  analysis  of  cost,  schedule, 
performance,  and  other  pertinent  financial  management  aspects 
for  the  Chief  of  Naval  Operations  and  the  Secretary  of  the  Navy 
before  DSARC  proceedings.  The  objective  is  to  test  the 
reasonableness  of  the  program  manager's  estimate. 

The  Naval  Material  Command  Cost  Analysis  Division  also  has 
a  small  staff  which  reviews  estimates  sent  up  by  the  commands. 
Its  review  is  also  aimed  at  testing  the  reasonableness  of  the 
program  office's  estimate. 
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CRITERIA  BASIC  TO  AN 
EFFECTIVE  ESTIMATING  PROCESS ^ 


CLEAR  IDENTIFICATION  OF  TASK 

To  prepare  a  cost  estimate,  the  estimator  must  be  provided 
with  the  system  description,  ground  rules  and  assumptions,  and 
technical  and  performance  characteristics  of  the  system.  A 
well  defined  system  description  which  specifies  conditions  and 
constraints  of  the  estimate  is  essential  in  clearly  identifying 
the  scope  of  the  estimate  and  documenting  how  it  was  prepared. 

BROAD  PARTICIPATION  IN 

PREPARING  ESTIMATES 


The  acquisition  of  a  major  weapon  system  involves  many  DOD 
and  contractor  organizations  in  deciding  mission  need  and 
requirements,  and  defining  performance  parameters,  force  struc¬ 
tures,  and  other  system  characteristics.  The  cost  estimate 
should  ensure  that  all  organizations  which  have  had  an  input 
into  the  system  design  have  participated  in  preparing  the  cost 
estimate.  Each  organization  should  have  had  its  data  indepen¬ 
dently  verified  for  accuracy  and  completeness  and  have  cost 
controls  in  place  to  ensure  the  reliability  of  its  data. 

AVAILABILITY  OF  VALID  DATA 

Numerous  sources  of  data  are  available  to  the  cost  estima¬ 
tor.  These  data  sources  vary  in  reliability.  A  principal 
source  is  the  historical  data  base  from  which  cost  estimators 
project  costs  of  new  systems  from  previously  similiar  or 
comparable  systems.  The  estimator  should  use  care  in 
determining  whether  such  data  is  suitable  for  the  purposes 
intended.  The  data  should  reflect  current  cost  trends  and  be 
directly  related  to  the  systems  performance  characteristics  and 
specifications. 

STANDARDIZED  STRUCTURE  FOR  ESTIMATES 

DOD  Military  Standard  881-A  provides  a  standard  method, 
called  a  work  breakdown  structure,  for  dividing  the  acquisition 
effort  into  specific  work  packages  peculiar  to  a  type  of  system. 
The  identification  of  these  work  packages  becomes  more  detailed 
as  the  system  progresses  through  the  acquisition  cycle. 


^These  criteria  are  from  GAO's  July  24,  1972  report  entitled. 
Theory  and  Practice  of  Cost  Estimating  for  Major  Acquisitions. 
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The  work  breakdown  structure  ensures  that 

— estimates  can  be  related  to  the  total  program, 

— relevant  cost  categories  are  not  omitted, 

— the  estimate  can  be  refined  as  the  system  design  becomes 
more  defined,  and 

— estimates  for  similar  types  of  systems  can  be  compared  by 
estimators  and  decisionmakers. 

An  estimate  derived  from  the  work  breakdown  structure 
assists  management  in  monitoring  and  directing  diverse  project 
activities  being  contracted  by  the  services  and  the  contractors. 

PROVISION  FOR  PROGRAM  UNCERTAINTIES 

One  of  the  most  difficult  and  often  criticized  aspects  of 
cost  estimating  concerns  identifying  uncertainties  and 
developing  a  realistic  allowance  for  their  cost  impact.  Work 
objectives  should  be  divided  into  knowns  and  unknowns  and 
provisions  made  for  their  resolution. 

RECOGNITION  OF  INFLATION 


Economic  changes  over  the  period  of  a  system's  development 
and  acquisition  can  have  a  significant  effect  on  the  cost  to 
develop,  produce,  and  operate  the  weapon  systems.  It  is 
important  that  inflation  be  recognized  and  realistically 
provided  for  if  estimates  for  total  program  costs  are  to  be 
valid. 

RECOGNITION  OF  EXCLUDED  COSTS 


Weapon  systems  cost  estimates  should  contain  provisions  for 
all  costs  associated  with  that  weapon  system.  If  major  costs 
have  been  excluded  from  an  estimate  or  included  under  another 
category,  it  is  important  that  the  estimator  disclose  this 
information  and  include  its  rationale. 

INDEPENDENT  REVIEW  OF  ESTIMATES 

An  independent  review  of  a  cost  estimate  is  crucial  to  the 
establishment  of  confidence  in  the  estimate.  The  independent 
estimator  must  examine  the  original  estimate  and  verify,  modify, 
and  correct  it  as  necessary  to  ensure  completeness,  consistency, 
and  realism  of  the  information  contained  in  the  cost  estimate. 

REVISION  OF  ESTIMATES  WHEN 
SIGNIFICANT  PROGRAM  CHANGES  OCCUR 

It  is  important  that  cost  estimates  be  updated  to  reflect 
changes  because  changes  in  the  system's  design  requirements 
drive  the  cost.  Large  changes  in  the  cost  of  an  acquisition 
significantly  influence  decisions  to  continue,  modify,  or 
terminate  a  program. 
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DESCRIPTIONS  OF  SYSTEMS  REVIEWED 

ARMY 

Apache  attack  helicopter  (AH-64) 


APACHE  HELICOPTER 


The  Apache  attack  helicopter  is  a  twin-engine,  four-bladed 
helicopter  operated  by  a  tandem-seated  crew  of  two.  The  pilot 
is  located  in  the  rear  cockpit,  and  the  copilot/gunner  is  in  the 
forward  position  where  concentrating  on  detecting,  engaging,  and 
destroying  enemy  targets  with  the  laser-guided  Hellfire  missile, 
2.75-inch  aerial  rockets,  and  30-mm  chain  gun  can  be  enhanced. 

To  perform  target  sighting,  the  Apache  is  equipped  with  a 
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special  target  acquisition  designation  sight  which,  along  with 
the  pilot  night  vision  system,  also  enhances  the  crew's  ability 
to  navigate  and  attack  targets  during  darkness  and  in  conditions 
of  limited  visibility. 

The  Army  has  developed  the  Apache  and  its  primary  armament, 
the  Hellfire,  to  be  its  principal  aerial  antiarmor  weapon  for 
the  mid-1980s  and  beyond.  Their  basic  mission  is  to  support 
ground  forces  by  destroying  enemy  tanks  and  other  ground  targets 
from  the  air. 
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Hellfire  Missile  System 


HELLFIRE  MISSILE 


The  Hellfire  is  an  antitank  missile  developed  specifically 
for  the  AH-64  Apache  helicopter.  The  missile  is  laser  guided; 
its  seeker  homes  onto  laser  energy  reflected  from  a  designator 
onboard  the  Apache  or  other  helicopters  accompanying  the  Apache 
or  located  on  the  ground.  According  to  the  Army,  several 
Hellfires  can  be  launched  in  rapid  succession,  enabling 
simultaneous  engagements  of  more  than  one  target. 
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Bradley  Fighting  Vehicle  Systems 


BRADLEY  FIGHTING  VEHICLE  SYSTEM 


The  objective  of  the  Bradley  Fighting  Vehicle  Systems 
program  is  to  develop  and  field  vehicles  that  normally,  in 
combat  operations,  will  be  employed  as  part  of  a  combined  arms 
team  with  the  Ml  Abram  Tank.  Included  in  the  Bradley  program 
are  the  Infantry  Fighting  Vehicle,  formerly  the  Mechanized 
Infantry  Combat  Vehicle,  and  the  Cavalry  Fighting  Vehicle, 
formerly  the  Armored  Reconnaissance  Scout  Vehicle.  They  are 
companion  vehicles  to  the  Ml  tank  in  close  combat  offensive  and 
defensive  operations.  The  Infantry  Vehicle  provides  mounted 
fighting  capability  for  a  9-person  (soon  to  be  10-person) 
mechanized  infantry  squad.  It  is  equipped  with  six  firing  port 
weapons,  a  25-mm  automatic  gun,  a  7.62  coaxial  machinegun,  and  a 
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smoke  grenade  launcher.  The  Calvary  Vehicle,  with  five  crew 
members,  will  serve  as  the  scout  vehicle  in  mechanized  infantry 
and  armor  battalions  and  in  armored  cavalry  squadrons. 

The  Infantry  Vehicle  will  replace  selected  M113A1  armored 
personnel  carriers  in  designated  mechanized  infantry 
battalions.  The  Calvary  Vehicle  will  replace  selected  M113Als 
and  improved  tube-launched,  optically-tracked ,  wire-guided  TOW 
missile  vehicles. 
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AIR  FORCE 

B-IB  long-range  combat  aircraft  system 


B-1B  BOMBER 


The  B-IB  is  being  designed  to  carry  out  manned  bomber 
operations  throughout  the  spectrum  of  conflict.  Its  primary 
mission  is  as  a  penetrating  bomber.  Other  missions  which  the 
B-IB  is  to  perform  include  the  support  of  tactical  land  and  air 
warfare,  naval  warfare,  and  theater  nuclear  warfare. 
Consequently,  the  B-IB  is  to  provide  flexibility  with 
an  inherent  capability  to  carry  a  wide  variety  of  weapons  over 
long  distances. 


77 


APPENDIX  V 


APPENDIX  V 


The  B-IB  uses  the  B-1A  aerodynamic  shape  and  structures  as 
well  as  many  systems  designed  for  the  B-IA.  To  permit  higher 
gross  take-off  weights,  the  B-IB  will  have  stronger  landing  gear 
than  the  B-1A. 

Communications  and  traffic  control  systems  on  the  B-IB  are 
essentially  the  same  as  those  incorporated  on  the  last  B-IA 
built.  The  B-IB  avionics  systems  are  to  be  improved  to 
accommodate  expanded  missions.  Offensive  avionics  systems 
incorporate  Improved  equipment  like  that  being  installed  in 
B-52s.  It  will  also  have  a  multimode  radar  derived  from 
equipment  being  installed  in  F-16  aircraft.  The  defensive 
systems  include  an  expanded  version  of  the  ALQ-161  electronic 
countermeasures  system  originally  designed  for  the  B-IA. 

The  B-IB  weapon  bays  will  have  the  capability  to  carry  all 
types  of  mission  required  weapons  as  well  as  additional  fuel. 

The  aircraft  is  operated  by  four  crew  members. 
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T-46A  trainer  aircraft  system 


T-46A  TRAINER  AIRCRAFT 


The  T-46A  is  a  development  and  acquisition  effort  to 
replace  the  T-37  aircraft.  The  T-46A  aircraft  is  to  provide  the 
Air  Force  primary  training  capability  beyond  1986.  The  T-46A 
aircraft  is  to  provide  significant  improvements  in  performance, 
maintainability,  and  noise  pollution  over  the  T-37  aircraft. 

The  T-46A  aircraft  is  considered  to  be  within  the  state  of  the 
art  with  low  complexity  in  its  components. 
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NAVY 

LSD-41 


LANDING  SHIP  DOCK  (LSD)-41 


The  LSD-41  program  is  designed  to  replace  the  LSD-28 
class.  The  ship  design  is  a  modified  repeat  LSD-36  design  which 
the  Navy  emphasizes  is  technically  proven,  using  simple, 
unsophisticated  systems.  The  ship  is  to  displace  about  15,745 
tons  fully  loaded  and  be  diesel  powered. 


80 


APPENDIX  V 


APPENDIX  V 


LAMPS  MK  III 


LIGHT  AIRBORNE  MULTI-PURPOSE  SYSTEM  (LAMPS)  MK  III  HELICOPTER 


LAMPS  MK  III  is  a  computer- integrated  ship  and  helicopter 
system  designed  principally  for  antisubmarine  warfare  with 
secondary  mission  capabilities  of  antiship  surveillance  and 
targeting,  search  and  rescue,  medical  evacuation,  and  logistics 
support.  LAMPS  MK  III  is  to  be  deployed  aboard  cruisers, 
destroyers,  and  frigates.  The  helicopter,  designated  the  SH-60B 
Seahawk,  is  a  derivative  of  the  Army's  UH-60A  Black  Hawk  troop 
assault  helicopter. 
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OTHER  REPORTS  OR  STUDIES  RELATING  TO  POD'S 

COST  ESTIMATING  AND  REPORTING  PROCESS 

(1)  Majors  Edwin  M.  Lewis,  and  Eugene  D.  Pearson,  (Air  Force), 
The  Air  Force  Cost  Estimating  Process;  The  Agencies 

Involved  and  Estimating  Techniques  Used  (Wright-Patterson 
Air  Force  Base) ,  Ohio;  Air  Force  Institute  of  Technology, 
June  1977. 

Within  the  Air  Force  cost  estimating  process,  cost 
estimates  are  developed  in  an  atmosphere  of  optimism.  The 
cost  estimator  is  provided  organizational  motivation  to 
develop  an  estimate  which  can  be  used  to  advocate  the  new 
weapon  system  to  DOD  and  the  Congress.  The  individual 
estimator  is  hampered  in  efforts  to  develop  estimates  by  the 
lack  of  an  adequate,  standardized  data  base;  lack  of 
feedback  on  the  accuracy  of  the  estimates;  and  little  or  no 
feedback  on  how  changes  have  effected  prior  estimates. 

( 2 )  Inaccuracy  of  Department  of  Defense  Weapons  Acquisition  Cost 

Estimates ,  House  Committee  on  Government  Operations,  96^^ 

Congress,  1st  Session,  Report  No.  96-656,  November  16,  1979. 

On  November  15,  1979,  the  Committee  on  Government 
Operations  approved  and  adopted  the  Legislation  and  National 
Security  Subcommittee's  report  entitled  Inaccuracy  of 
Department  of  Defense  Acquisition  Cost  Estimates.  Some  of 
the  study's  findings  were  as  follows: 

— Major  weapon  system  cost  estimates  are  not  accurate  and 
do  not  provide  the  Congress  with  the  information  needed 
to  effectively  evaluate  DOD  budget  requests. 

— The  initial  planning  estimate  has  been  approximately 
100  percent  below  the  actual  cost  of  major  systems, 
while  the  later,  more  refined  development  estimate  has 
been  approximately  50  percent  below  procurement  costs. 

— The  major  reasons  cited  by  the  military  for  cost 

variances  are;  economic  inflation  changes,  30  percent; 
quantity  changes,  28  percent;  and  schedule  changes,  16 
percent.  Also,  the  military  acknowledges  an  "actual 
estimating  error"  of  12  percent  which  amounts  to  over 
$10  billion. 

— Because  major  weapon  systems  cost  estimates  have  not 
proven  accurate  over  time,  the  quantities  of  weapons 
procured  have  had  to  be  reduced,  thereby  cutting  into 
the  national  security  potential. 
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— Because  of  continued  underestimating,  there  has  been 
created  a  so-called  "bow  wave"  effect,  which  means  that 
weapons  procurement  will  probably  have  to  be  further 
cut  in  the  future  because  of  insufficient  funds. 

— The  cost  estimating  process  used  by  the  services  does 
not  parallel  the  service  and  DSARC  process,  which  may 
contribute  to  the  difficulty  in  developing  more 
accurate  estimates. 

— Insufficient  staff  resources  within  the  three  services 
and  within  OSD  may  contribute  to  the  consistently 
inaccurate  cost  estimates  for  major  weapon  systems. 

Discussions  during  these  hearings  also  revealed  that 
DOD  continues  to  use  unrealistic  inflation  rates.  Further, 
compounding  the  problem  is  DOD's  practice  of  attributing  to 
"economic  change"  some  of  the  cost  increases  due  to  delays 
in  the  production  schedule  of  a  weapon  system,  in  addition 
to  actual  increase  due  to  the  unanticipated  economic  rate 
of  inflation. 

The  Committee  report  concluded  the  following: 

It  is  the  Committee's  intention  to  demonstrate 
that  the  Congress  must  have  available  reliable  and 
accurate  cost  information  at  critical  points  in  the 
weapon  systems  authorization  and  appropriations  process 
if  the  American  public  is  to  receive  maximum  benefit 
from  the  expenditure  of  their  defense  dollars.  The 
record  is  clear  that  the  Congress  has  not  been  getting 
that  information. 

( 3 )  Department  of  the  Army,  Office  of  the  Auditor  General,  Cost 

Discipline  Advisory  Committee  Special  Review,  Audit  Report 

No.  HQ  82-701  ,  10/28/81  . 

Inflation  indexes  used  by  Army  program  managers  to 
estimate  program  costs  are  based  on  different  methodologies 
and  should  be  assessed  for  impact  on  programs. 

Inflation  indexes  on  the  M-1  tank  program  were 
overstated  when  compared  to  the  actual  rate  of  inflation 
incurred.  These  overstated  rates  had  the  effect  of  masking 
real  cost  increases  on  the  program.  The  M-1  cost  estimates 
are  overly  optimistic  and  were  primarily  based  on  contractor 
estimates . 
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(4)  U.S.  Congress,  House  Committee  on  Armed  Services,  Special 
Panel  on  Defense  Procurement  Procedures  Report,  Weapons 
Acquisition  Policy  and  Procedures;  Curbing  Cost  Growth, 

97th  Congress,  1st  Session,  Committee  Print  No.  13, 

February  12,  1982. 

Specifically,  the  panel  found  that  the  factors  contributing 
to  cost  growth  are  unrealistic  inflation  estimates,  poor 
cost  estimates,  program  stretchouts,  changes  in 
specifications,  inadequate  budgeting,  high  risk  system 
design,  poor  management,  and  lack  of  competition. 

The  panel  found  that  each  year  DOD  continues  to  base 
its  budget  request  on  unrealistically  low  inflation 
estimates  established  by  0MB.  In  fact,  DOD's  own  analysis 
of  recent  cost  growth  data  indicates  that  approximately  30 
percent  of  the  cost  growth  experienced  by  the  47  major 
weapon  systems  reported  in  SARs  is  due  to  low  inflation 
estimates.  During  one  of  the  panel's  hearings.  Deputy 
Secretary  Carlucci,  referring  to  DOD's  recent  management 
initiatives  to  improve  the  acquisition  process,  indicated 
that,  "...  budgeting  for  inflation  is  perhaps  the  most 
difficult  of  all  the  initiatives."  Carlucci  continued, 

"I  can  claim  some  progress  in  a  lot  of  other 

initiatives,  but  at  this  point  I  can't  claim  that 

we  have  solved  that  problem." 

Another  significant  contributor  to  cost  growth  is 
erroneous  cost  estimating  by  contractors.  The  panel  found 
several  examples  of  poor  estimating  by  contractors  during 
its  investigation.  For  example,  during  the  panel's  case 
study  of  the  Black  Hawk  helicopter  program,  it  was 
determined  that  for  the  period  1977-79  the  contractor 
underestimated  the  man-hours  required  to  produce  the  first 
163  Black  Hawk  helicopters  by  over  54  percent.  This  means 
that  instead  of  the  97,200  man-hours  initially  estimated, 
the  true  requirement  was  approximately  150,077  man-hours, 
which  resulted  in  a  significant,  unanticipated  cost  growth. 
This  is  not  an  isolated  example.  The  panel's  record  is 
replete  with  discussions  of  poor  estimating,  and  it  is  clear 
that  such  errors  are  major  contributors  to  cost  growth. 
Incentives  are  required  to  force  more  accuracy  and  realism 
into  estimates.  The  panel  record  clearly  documents 
contractors'  propensity  for  competitive  optimism — "buying 
in. " 


Among  the  panel's  finding  is  that  SAR  is  inadequate  in 
its  reporting  on  major  weapon  systems  to  the  Congress,  thus 
inhibiting  proper  oversight.  The  SAR  system  does  not 
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consistently  provide  timely  and  complete  information.  The 
present  SAR  system  provides  quarterly  updates  that  do  not 
always  reflect  substantive  fact-of-life  changes  that  may 
have  occurred  in  a  program  since  the  December  31  SARs. 

Although  Secretary  Carlucci  testified  that, 

"We  do  have  the  CAIG  [Cost  Analysis  Improvement 
Group] ,  as  you  know,  which  works  with  the  DSARC 
to  provide  the  independent  cost  analysis," 

the  record  is  not  clear  on  whether  Cost  Analysis 
Improvement  Group  cost  analyses  actually  influence  the 
final  cost  estimates  which  appear  in  defense  budget 
proposals.  Mr.  Augustine  testified  that, 

"They  [CAIG]  have  some  competent  people,  but  I 
would  fault  their  efforts  in  two  regards:  One, 
their  methodology  is  not  sufficiently  accurate  to 
give  high-cost  [highly  accurate]  cost  estimates; 
and  secondly,  they  are  too  seldom  listened  to. 

If  we  had  listened  to  them  more  carefully  in  the 
past,  we  might  not  be  sitting  here  today." 

(5)  Joseph  T.  Kammerer,  Estimating  in  the  '80s — Special 
Section,  Concept.  The  Journal  of  Defense  System  Acquisition 
Management,  spring  1981,  Volume  4,  Number  2. 

Many  cost  analysts  shy  away  from  projecting  the  impact 
of  the  economy  on  future  defense  expenditures.  The  excuse 
is  often  given  that  such  projections  are  too  subjective,  too 
uncertain,  and  would  be  viewed  as  guesswork.  The  current 
procedure  also  allows  us  to  conveniently  place  the  blame  for 
cost  growth  on  inflation,  an  economic  force  beyond  our 
control . 

Whenever  a  Navy  weapon  systems  program  has  significant 
cost-growth  problems,  the  questions  from  top  management  are, 
"What  happened?  Where  did  we  go  wrong  in  estimating  the 
costs?"  The  answers  are  often  not  readily  available  and  the 
reason  is  the  lack  of  good  cost  estimating  documentation. 

To  track  cost  estimates,  good  documents  are  essential.  Even 
when  documents  are  fairly  good,  the  question  can  often  not 
be  answered  satisfactorily  because  cost  estimators  do  not 
track  cost  performance  very  well. 

( 6 )  American  Defense  Preparedness  Association,  Cost  Discipline 

Report ,  Chicago,  Illinois,  July  21  to  22,  1982. 

Cost  estimates  on  weapon  systems  development  programs 
are  not  well  done,  are  not  updated  with  sufficient 
frequency,  and  are  characterized  by  being  overly  optimistic. 
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particularly  at  the  early  stages  of  the  program.  Many  of 
these  problems  stem  from  the  competitive  nature  of  the  DOD 
budget,  defense  industry  competition,  and  the  contracting 
process. 

Although  on  the  surface,  the  Army,  OSD,  and  the 
Congress  want  to  know  the  real  costs,  the  competitive 
environment,  both  within  the  budget  process  and  with  the 
defense  industry,  creates  an  atmosphere  that  results  in  very 
optimistic  and  high-risk  programs  with  questionable  cost 
realism. 

(7)  Dr.  Walter  B.  LaBerge,  Defense  Acquisition;  A  Game  of 
Liar ' s  Dice?  Concepts,  The  Journal  of  Defense  Systems 
Acquisition  Management,  winter  1982. 

Prior  attempts  at  improving  DOD  acquisition  have 
centered  on  improving  the  process  without  attacking  the 
strong  environmental  motivations  that  make  people  behave  the 
way  they  do.  The  DOD  procurement  system  performs  in  a 
nonoptimal  way  with  large  overruns  and  delays  because  the 
environment  forces  that  behavior.  The  DOD  acquisition 
system  is  very  much  like  the  barroom  game  of  "lair's  dice." 
In  that  game,  winning  comes  from  concealing  the  true  facts 
(e.g.,  the  roll  of  one's  dice)  and  by  asserting  not  what  is, 
but  rather  what  might  be. 

At  the  outset  of  a  program,  the  DOD  bid  process 
encourages  substantial  contractor  overoptimism  in  technical 
accomplishment,  in  schedule,  and  in  cost.  The  management 
level  above  the  program  manager  is  itself  swept  up  in  its 
own  liar's  dice  game.  The  figure  of  merit  by  which  senior 
officials  are  scored  is  how  well  they  can  convince  their 
military  and  civilian  superiors  that  their  own  program 
should  be  funded  instead  of  someone  else's.  To  do  this, 
they  also  need  programs  which  can  be  promised  quickly  and  at 
low  cost.  In  sum,  no  one  benefits  from  being  conservative. 
Everyone  benefits  by  being  unrealistic.  All  the  incentives 
are  on  selling. 

(8)  George  W.  S.  Kuhn,  A  Defense  Strategy  for  the  80s, 
Washington,  DC:  Heritage  Foundation,  January  21,  1983. 

No  rationale  whatever  exists  for  DOD  continuing  to 
ignore  future  program  changes,  which  are  the  principal 
cause  of  acquisition  cost  growth.  More  direct  control  must 
be  exercised  over  cost  estimates  and  cost  escalation.  Cost 
overruns  must  be  penalized  by  enforcing  standards  of  cost 
escalation.  The  original  quantity  requirement  might  serve 
as  an  upper  limit  on  procurement;  the  original  program  cost 
estimate  could  be  an  upper  spending  limit.  Preferably, 
some  range  of  growth  above  the  original  cost  estimate 
should  be  allowed.  The  range  might  differ  with  different 
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kinds  of  equipment — tracked  vechicles  versus  aircraft,  for 
example.  The  range  might  specify  allowable  limits  for 
different  kinds  of  cost  increases:  a  greater  margin  could 
be  allowed  for  technical  changes  to  meet  original 
performance  goals;  the  margin  for  changes  to  add 
performances  capability  could  be  more  limited.  Whatever 
the  standard  chosen,  the  limit  must  be  strictly  enforced. 
The  Army's  Fighting  Vehicle  System  long  ago  violated  any 
reasonable  cost  growth  margin:  it  is  now  more  than  2,500 
percent  over  estimate. 

Budgeting  practices  are  altogether  unrealistic  in 
their  attempts  to  plan  for  and  manage  cost  escalation. 

Cost  projections  are  consistently  and  optimistically 
understated.  During  the  program's  acquisition,  no  adequate 
reference  is  made  to  the  actual  cost  escalation  experience 
of  other  similar  programs.  Even  the  program's  own  history 
of  cost  change  is  seldom  taken  as  a  guide.  Finally, 
budgets  are  assumed  to  increase  at  steady  rates  (high  or 
low) ,  which  has  never  proved  true  even  in  the  best  budget 
times . 


The  result  of  unrealistic  cost  and  budget  projections 
is  an  enormous  upward  pressure  on  budgets  from  within 
ongoing  programs.  Put  differently,  almost  every  program  in 
the  DOD  acquisition  process  needs,  at  any  given  time, 
greatly  expanded  funds  just  to  accomplish  current  goals. 
Moreover,  the  overall  budget  is  subjected  to  more  such 
pressure  with  every  addition  of  a  new  program.  These 
internal  budget  pressures  are  of  such  proportions  that  even 
generously  increasing  budgets  cannot  keep  up.  Any  new 
funds  are  soaked  up  with  little  or  no  new  output. 

(9)  Air  Force,  Affordable  Acquisition  Approach  report, 

February  9,  1983. 

A  study  of  old  and  new  weapon  systems  acquisitions 
concludes  that  it  is  taking  longer  and  costing  more  to 
acquire  major  Air  Force  systems.  In  addition,  the  report 
states  that  some  key  cost  management  tools  are  needed  such 
as  developing  a  more  likely  cost  estimate  that  includes 
future  growth,  unknowns,  and  so  on.  The  report  concludes 
that  once  the  most  likely  cost  is  established,  the  Air 
Force  must  program  and  fund  to  that  level,  and  that  the 
Air  Force  must  ensure  realistic  inflation  rates  are  used  in 
program  estimates. 
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(10)  Congressional  Budget  Office,  Special  Study,  A  Review  of 
the  Department  of  Defense  December  31,  1982,  SARs, 

August  1983. 

Some  of  the  findings  of  the  Congressional  Budget 
Office  study  of  the  December  31,  1982,  SARs  for  62  weapon 
systems  are  presented  below.  The  Congressional  Budget 
Office  made  an  analysis  of  cost  changes  reported  by  DOD 
for  the  fourth  quarter  of  1982,  for  the  1982  calendar  year 
as  a  whole,  and  over  the  years  from  1977  to  1982. 

SARs  are  very  useful  for  monitoring  cost  changes  and 
other  developments  in  weapons  acquisition  programs  and  for 
providing  rough  indicators  of  overall  cost  growth.  In 
several  respects,  however,  the  SARs  continue  to  contain 
incomplete,  inaccurate,  and  conflicting  information. 

There  are  numerous  indications  that  the  latest  SAR 
data  may  not  reflect  the  ultimate  acquisition  cost. 

The  cost  estimates  for  13  systems  excluded  at  least 
$40.8  billion  in  program  costs  that  were  footnoted  in  SARs 
or  reported  in  other  DOD  budget  documents,  such  as  the 
congressional  data  sheets.  The  Congressional  Budget  Office 
believes  that  these  costs  should  be  included  in  the  SAR 
estimates.  Doing  so  would  raise  the  December  1982 
estimated  costs  for  the  13  systems  by  13.8  percent.  Of  the 
13  systems,  3  were  Air  Force  programs,  9  were  Navy 
programs,  and  1  was  an  Army  program.  For  the  F-15  and  F-16 
programs,  for  example,  the  Air  Force  did  not  report  almost 
$14  billion  for  procurement  of  additional  aircraft.  The 
Navy  did  not  report  over  $4  billion  of  military 
construction  costs  for  the  Trident  submarines.  The  Navy 
also  did  not  include  nearly  $6  billion  of  CVN  (Nuclear 
Aircraft  Carrier)  procurement  costs  for  a  carrier  in  1988 
and  advance  procurement  in  1986-88  for  later  ships. 

The  services  used  different  inflation  rates  in  1982  to 
estimate  the  procurement  costs  of  aircraft  and  missiles. 

The  use  of  different  inflation  rates  for  1982  could 
significantly  affect  estimates  of  future  program  costs 
because  the  differences  would  be  compounded  over  the 
years.  Moreover,  the  use  of  two  different  rates  for 
identical  systems  or  similar  systems  made  by  the  same  prime 
contractor  suggests  that  some  programs  are  overfunded  or 
underfunded . 
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(11)  Congressional  Research  Service,  Cost  Overruns  in  Major 
Weapon  Systems;  Current  Dimensions  of  a  Longstanding 

Problem,  Report  No.  83-194  F,  October  15,  1983. 

Among  the  factors  widely  considered  to  be  most 
responsible  for  cost  overruns  is  the  practice  of  making 
unrealistically  low  initial  cost  estimates.  In  testimony 
before  the  Senate  Committee  on  Governmental  Affairs  on 
March  23,  1983,  Deputy  Secretary  of  Defense  W.  Paul  Thayer 
stated  that  "unrealistic  cost  estimating  is  a  major  cause 
of  cost  growth."  He  went  on  to  explain  that: 

"In  the  past  we  have  been  overly  optimistic  about 
the  projected  outyear  costs  of  programs.  The 
cost  growth  that  results  from  this  optimism 
produced  increasing  instability,  stretch-outs, 
and  more  cost  increases.  It  is  a  vicious  cycle 
and  a  difficult  one  to  reverse."^ 

Indeed,  it  is  widely  acknowledged  that  program  costs 
are  often  purposely  underestimated  either  because  the 
contractors  are  lowering  their  cost  estimates  to  win  a 
contract  with  hopes  of  recovering  costs  on  follow-on 
contracts  (a  practice  known  as  "buying-in")  or  because  DOD 
is  forcing  a  program  to  fit  available  funding  rather  than 
providing  the  funding  it  takes  to  do  the  job.^ 


^Deputy  Secretary  of  Defense.  Internal  memorandum  to  the 
secretaries  of  the  military  departments  and  others  on  the 
subject  of  improving  the  acquisition  process;  with  attached 
recommendation  and  issues  for  decision,  April  30,  1981. 
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OUR  PAST  REPORTS  AND  TESTIMONY  RELEVANT 
TO  THE  POD  COST  ESTIMATING 
AND  REPORTING  PROCESS 


1 .  Status  of  Major  Acquisitions  as  of  September  30 ^  1981; 

Better  Reporting  Essential  to  Controlling  Cost  Growth 

(MASAD-82-24,  4/22/82). 

Agencies'  cost,  schedule,  and  quantity  data  show  that 
the  initial  estimates  and  planning  data  provided  to  the 
Congress  vary  greatly  from  current  estimates.  The  cost  of 
137  defense  programs  increased  a  total  of  $243.1  billion, 
or  132  percent  over  the  congressional  budget  estimates, 
that  is,  total  estimated  cost  of  projects  provided  the 
Congress  with  initial  budget  justifications.  Major  reasons 
cited  for  cost  growth  included 

— inflation  underestimated  and/or  not  included  in  the 
planning  estimate; 

— inaccurate  estimates;  and 

— changes  in  scope/quantity,  engineering,  schedule,  and 
mission . 

Over  the  years,  we  have  reported  on  the  problem  of  cost 
growth,  the  need  to  improve  existing  reporting  systems,  and 
the  need  for  establishing  a  reporting  system  to  include  all 
major  civil  acquisitions  and  those  major  defense  programs 
not  on  DOD ' s  SAR  system. 

2.  Budgetary  Pressures  Created  by  the  Army's  Plans  to  Procure 

New  Major  Weapon  Systems  Are  Just  Beginning  (MASAD-82-5, 
10/20/81 ) . 

Procurement  costs  for  14  major  Army  weapon  systems 
increased  by  30  percent  from  $52.2  billion,  to  $68.1 
billion,  over  the  1981-85  plan.  This  was  due  to  a 
combination  of  factors;  (1)  the  effects  of  deliberate 
production  stretchout,  which  include  incurring  more  fixed 
cost  than  planned  and  higher  inflation  associated  with 
greater  annual  spending  rates  in  future  years,  (2)  program 
cost  growth  due  to  underestimating,  and  (3)  the  use  of 
higher  inflation  estimates  than  had  been  used  previously. 
Early  production  contracts  were  budgeted  using  cost 
estimates  that  have  proven  to  be  optimistic. 
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3.  Improving  the  Weapon  Systems  Acquisition  Process 

(MASAD-81-29,  5/15/81). 

A  compilation  of  concerns  regarding  the  weapon  systems 
acquisition  process  is  given.  These  represent  our  opinions 
based  on  the  many  reviews  we  have  made  over  the  past  several 
years.  The  services  should  be  instructed  to  prepare  their 
cost  estimates  conservatively.  Too  much  optimism  has  crept 
into  past  estimates.  The  recent  sharp  rise  in  the  cost 
estimates  of  several  acquisition  programs  shows  that 
sufficient  realism  was  lacking  in  developing  these 
estimates.  A  more  critical  review  of  the  cost  estimates 
prepared  for  presentation  to  DSARC  should  be  made. 

4.  Cost  Growth  in  Major  Weapon  Systems  (B-163058,  3/26/73). 

Costs  of  45  systems  covered  by  DOD  SARs  at  June  30, 
1972,  increased  by  some  $31.5  billion,  or  39  percent  over 
planning  estimates  and  20  percent  over  development 
estimates.  One  reason  for  this  cost  growth  was  inaccuracy 
in  estimating  by  both  competing  contractors  and  military 
service  representatives.  Both  the  competing  contractors  and 
service  representatives  are  strongly  motivated — the  one  to 
propose  and  the  other  to  accept  low  cost  estimates — to  win 
the  contract  and  to  gain  approval  against  competing  systems 
within  the  same  service  or  similar  systems  proposed  by 
another  service.  It  is  neither  wise  nor  equitable  to 
delegate  cost  estimates  for  new  development  to  competing 
contractors,  nor  to  base  the  government's  funding  plans  on 
such  estimates.  The  government  must  have  the  capability  to 
develop  reliable  cost  estimates. 

5 .  Theory  and  Practice  of  Cost  Estimating  for  Major 

Acquisitions  (B-163058,  7/24/72). 

Uniform  guidance  on  cost  estimating  practices  and 
procedures  which  would  be  the  basis  for  formulating  valid, 
consistent,  and  comparable  estimates  throughout  the  services 
was  lacking.  Each  service  issued  its  own  guidance  for  the 
estimating  function,  which  ranged  from  a  detailed  estimating 
manual  to  a  few  general  statements.  Guidance  was  often 
ignored  by  the  estimators. 

Cost  estimates  for  a  specific  system  frequently  are  a 
succession  of  revisions;  the  current  cost  estimate  is 
derived  by  refining  and  revising  the  preceeding  cost 
estimate.  Accurate  revision  of  both  the  original  and 
updated  cost  estimates  require  documents  showing  data 
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sources,  assumptions,  methods,  and  decisions  basic  to  the 
estimate.  In  virtually  every  system  we  reviewed,  documents 
supplying  such  information  were  inaccurate  or  werelacking. 
Among  the  resulting  difficulties  were: 

— Known  costs  had  been  excluded  without  adequate  or 
valid j  ustif ication . 

— Historical  cost  data  used  as  a  basis  for  computing 
estimates  were  sometimes  invalid,  unreliable,  or 
unrepresentative . 

— Inflation  was  not  always  included  or  uniformly 
treated  when  it  was  included. 

— Understanding  and  proper  use  of  the  estimates  was 
hindered . 

Readily  retrievable  cost  data  which  could  serve  as  a  base 
for  computing  cost  estimates  for  new  weapon  systems 
generally  were  lacking. 

We  recommended  that  the  Secretary  of  Defense  should 
develop  and  implement  guidance  for  consistent  and  effective 
cost  estimating  procedures  and  practices  throughout  DOD.  In 
developing  this  guidance,  the  Secretary  of  Defense  should 
consider  the  criteria  for  cost  estimating  set  out  in  this 
report.  Of  particular  importance  is  providing  for: 

— An  adequate  data  base  of  readily  retrievable  cost 
data. 

— Treatment  of  inflation. 

— An  effective  independent  review  of  cost  estimates, 
including  judgment  by  top  officials  as  to  the 
realism  of  the  cost  estimates  on  which  decisions  are 
based . 

— More  complete  documents  of  cost  estimates,  coupled 
with  a  requirement  for  an  adequate  feedback  of 
results,  to  provide  a  basis  for  comparing  costs 
achieved  with  those  estimated. 

6 .  Evaluation  of  Unit  Cost  Reports  Submitted  Under  Public  Law 

97-86,  Section  917  (MASAD-82-36 ,  5/10/82) 

We  reviewed  19  unit  cost  reports  submitted  to  the 
Congress  by  DOD  to  determine  the  completeness,  currentness, 
and  accuracy  of  the  data  presented.  Overall,  DOD  made  a 
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reasonably  successful  effort  to  comply  with  the  requirements 
of  submitting  unit  cost  reports.  However,  some  reports  were 
lacking  in  completeness  and  accuracy.  In  some  cases, 
program  estimates  did  not  include  all  costs,  were  not 
consistent  with  independent  estimates,  and  will  soon  be 
outdated  because  of  changes  and  uncertainties  in  the 
program. 

7.  The  B-1  Bomber  Program — A  New  Start  (GAO/MASAD-83-21 , 
4/13/83). 

The  B-1B  program  still  omits  known  program  costs,  for 
example,  independent  cost  estimates  prepared  by  OSD  and  Air 
Force  analysts  agree  on  $1.4  billion  in  additional 
acquisition  costs  not  included  in  the  estimate  provided  to 
the  Congress.  OSD  and  Air  Force  guidance  defining  program 
costs  permit  varying  interpretations  of  what  is  to  be 
included  in  major  acquisition  cost  estimates.  We  believe 
that  the  Congress  would  have  better  visibility  of  the 
acquisition  cost  if  all  related  costs  were  reported  in  one 
place.  We  recommended  that  all  acquisition  costs  related  to 
the  B-1B  program  be  provided  to  the  Congress  in  a  single 
package . 

8 .  Consistent  and  Uniform  Treatment  of  Inflation  Needed  in 
Program  Cost  Estimates  Provided  to  the  Congress  (PSAD-78-8, 
3/20/78) . 

0MB  and  agency  procedures  do  not  result  in  uniform 
treatment  of  expected  inflation  or  price  changes  in  the 
budget  and  cost  estimates  provided  to  the  Congress.  The 
Congress  should  require  that  0MB  develop  inflation  policy 
and  procedures  which  agencies  would  uniformly  apply  to 
annual  program  and  budget  estimates.  Uniform  criteria  would 
provide  comparable  program  data. 

9 .  A  Range  of  Cost  Measuring  Risk  and  Uncertainty  in  Major 

Programs:  An  Aid  to  Decisionmaking  (PSAD-78-12,  02/02/78 ) . 

Although  efforts  can  and  should  be  made  to  continuously 
improve  the  reasonableness  of  program  estimates,  over 
optimism  still  exist.  Problems  caused  by  inflation,  cost 
estimating,  and  program  uncertainty  are  further  complicated 
by  the  advocacy  or  optimistic  nature  of  many  estimates.  A 
major  program  within  an  agency  competes  with  other  agency 
programs  for  funds.  The  agency  must  compete  with  other 
federal  agencies  for  a  share  of  the  total  federal  budget. 
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Since  funds  are  limited,  program  proponents  tend  to  state 
the  most  favorable  estimate  which  reflects  the  assumption 
that  no  problems  will  arise  or  that  problems  will  have  only 
a  minimal  effect. 

1 0 .  Cost  Information  Presented  to  the  Congress  on  the  C-5B 

Aircraft  Program  (GAO/MASAD-83-5 ,  1 2/30/82 ) . 

The  C-5B  cost  estimate  of  $8.8  billion  supporting  the 
fiscal  year  1983  budget  was  understated  by  $700  million  due 
to  the  omission  of  certain  acquisition  costs  (ground 
support  equipment,  simulators,  technical  data,  etc.)  from 
its  estimate. 

1 1 .  F/A-18  Naval  Strike  Fighter;  Progress  Has  Been  Made  but 

Problems  and  Concerns  Continue  (MASAD-81-3,  2/18/8 1 ) . 

Although  there  has  been  some  cost  growth,  costs  have 
soared  primarily  because  of  the  inflation  impact.  Even 
though  substantial  increases  have  already  been  reported, 
the  current  program  cost  estimate  is  based  on  projected 
escalation  rates  which  are  consistently  lower  than  those 
projected  by  industry.  If  actual  escalation  rates  continue 
to  be  higher  than  rates  used  by  DOD,  program  cost  estimates 
will  continue  to  be  understated. 

1 2 .  Impediments  to  Reducing  the  Costs  of  Weapon  Systems 

(PSAD-80-6,  11/8/79). 

We  found  that  the  design-to-cost  concept  was  not 
closely  followed  and  the  departures  included  ( 1 )  failure  to 
establish  the  cost  data  base  needed  to  establish  cost 
performance  estimating  relationships  relevant  to 
design- to-cost  objectives,  goals,  and  decisions  and  (2) 
overemphasis  on  controlling  the  more  immediate  and  visible 
acquisition  costs  than  the  more  substantial  life-cycle 
costs . 

13.  Comparative  Life-Cycle  Cost;  A  Case  Study  (PAD-78-21, 
8/16/78) . 

A  case  study  of  the  costs  for  two  Army  tanks,  XM-1  and 
M60A3,  demonstrated  the  importance  of  life-cyle  cost 
comparisons.  Comparing  only  acquisition  costs,  the  XM-1  is 
twice  as  expensive  as  the  M60A3.  When  life-cycle  costs 
(totals  for  buying,  operating,  and  maintaining  tanks) 
during  a  20-year  period  are  considered,  the  XM-1  costs  are 
about  20  percent  more  because  of  large,  nearly  equal 
maintenance  and  operation  costs  for  the  two  tanks.  Thus, 


94 


APPENDIX  VII 


APPENDIX  VII 


the  XM-1  need  be  only  20  percent  more  cost  effective,  and 
the  consensus  of  Army  studies  has  been  that  its  cost 
effectiveness  compared  to  the  M60A3  is  considerably  more 
than  this  percentage. 

14.  Review  of  Life-Cycle  Cost  Concept  (PSAD-78-74,  3/2/78). 

Actions  have  been  initiated  encouraging  the  use  of 
life-cycle  cost  considerations  in  decisionsmaking ,  but 
progress  has  been  slow.  Issuance  of  guidance  and  operating 
and  support  costs  accumulation  and  reporting  has  been 
spotty,  and  the  effect  of  initiated  actions  is  just  being 
recognized.  Issues  requiring  attention  are;  accountabil¬ 
ity  for  meeting  operating  and  support  cost  goals  is 
unclear,  assumptions  for  developing  operating  and  support 
estimates  are  not  updated  to  show  changes  in  program 
requirements,  operating  and  support  estimates  are  not 
reported  to  the  Congress,  costs  of  operating  and  support 
data  are  not  compared  to  benefits,  and  more  life-cycle  cost 
training  is  needed.  To  make  life-cycle  cost  a  viable 
factor  in  controlling  operating  and  support  costs,  the 
Secretary  of  Defense  should;  accelerate  efforts  to  provide 
uniform  and  standard  cost  definitions  and  cost  estimating 
guides,  hold  program  managers  accountable  for  operating  and 
support  costs,  require  systematic  updating  and  reporting  of 
life-cycle  cost  estimates,  and  provide  program  managers 
expert  cost  analysts  to  assist  in  life-cycle  cost  decisions 
and  trade-offs. 

1 5.  Life-Cycle  Cost  Estimating — Its  Status  and  Potential  Use  in 

Major  Weapon  System  Acquisitions  (PSAD-75-23,  December  30, 
1974) . 

The  life-cycle  cost  of  a  weapon  includes  the  cost  to 
acquire,  operate,  and  maintain  the  weapon  over  its  useful 
life.  Decisions  on  proceeding  with  a  weapon  system's 
development  have  generally  been  based  on  the  acquisition 
cost  of  the  weapon. 

DOD  must  overcome  its  reluctance  to  disclose  ownership 
cost  estimates.  Acquisition  costs  of  major  weapon  systems 
are  a  matter  of  record — they  are  published  in  SARs  prepared 
for  the  Congress  by  DOD  for  most  of  the  larger  systems — but 
service  officials  are  not  as  open  about  disclosing 
estimates  of  ownership  costs. 

In  an  environment  where  major  weapon  systems  about  to 
be  or  already  being  developed,  are  competing  with  each 
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Other  for  funds,  the  concentration  is  on  securing 
sufficient  funds  to  "get  the  program  off  the  ground"  rather 
than  directing  attention  to  the  operation  and  maintenance 
costs  which  will  not  require  funding  for  several  years. 

The  Congress  may  wish  to  institute  procedures  by  which 
the  Secretary  of  Defense  will  periodically  inform 
interested  committees  of  the  Congress  on  the  progress  being 
made  in  improving  DOD's  ability  to  estimate  ownership  costs 
and  furnish  such  estimates  after  necessary  improvements  are 
achieved . 

1 6 .  Need  for  More  Accurate  Weapon  System  Test  Results  to  Be 

Reported  to  the  Congress  (PSAD-79-46,  3/9/79 ) . 

To  overcome  the  problem  of  not  receiving  accurate 
information  it  needs  in  making  weapon  system  procurement 
decisions,  the  Congress  enacted  legislation  requiring  DOD 
to  submit  annual  reports  to  the  Congress  (congressional 
data  sheets).  Among  other  things,  these  documents  are  to 
include  results  of  operational  tests  and  evaluations  on 
major  systems  for  which  procurement  funds  are  requested. 

We  also  found  that  the  results  of  the  tests  and 
evaluations  of  10  Navy  and  5  Air  Force  weapon  systems 
contained  in  the  data  sheets  reviewed  were  incomplete, 
misleading,  and/or  outdated. 

We  recommend  that  the  Secretary  of  Defense  improve  the 
quality  and  timeliness  of  operational  test  results  reported 
in  congressional  data  sheets. 

17.  Statement  of  Walton  H.  Sheley,  Jr.,  Director,  Mission 
Analysis  and  Systems  Acquisition  Division,  before  the 
Senate  Committee  on  Governmental  Affairs,  on  DOD 
Acquisition  Issues,  March  23,  1983. 

"The  failure  to  develop  reliable  cost 
estimates  results  in  cost  growth  that  is 
built-in,  that  is,  cost  growth  that  could 
have  been  avoided  if  more  time,  attention, 
and  realism  was  used  in  developing 
estimates.  All  too  often  optimistic 
estimates  are  used  to  gain  approval  for 
acquisitions.  Once  a  decision  is  made  on 
the  basis  of  faulty  estimates,  it  may  take 
years  before  the  real  costs  surface." 
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18.  Statement  of  Walton  H.  Sheley,  Jr.,  Director,  Mission 
Analysis  and  Systems  Acquisition  Division,  before  the 
Special  Panel  on  Defense  Procurement  Procedures,  House 
Armed  Services  Committee  on  Weapon  System  Cost  Growth, 
October  22 ,  1981. 

"As  far  back  as  the  early  1970s,  GAO  has 
reported  that  both  planning  and  development 
cost  estimates  on  Federal  acquisitions  in 
many  cases  are  quite  optimistic  on  technical 
development  problems,  cost,  and  potential 
performance.  Recognizing  the  technical 
complexity  of  R&D  [Research  and  Development] 
projects,  we  believe  it  is  extremely 
important  that  adequate  project  definition  be 
performed  to  provide  as  accurate  and  reliable 
an  estimate  of  schedule  milestones  and  total 
project  cost  as  possible. 

"The  desire  of  program  advocates  to  sell  the 
program  to  both  agency  management  and  the 
Congress  with  low-cost  estimates  and  high 
expectations  for  solving  technical  problems  is 
understandable.  After  all,  the  vitality  of  an 
agency  depends  to  a  large  extent  on  new 
program  starts.  But  this  must  be  balanced 
against  the  need  for  as  realistic  appraisals 
as  possible  of  the  potential  resources 
needed.  Recent  testimony  by  DOD  recognizes 
that  unrealistically  low  contractor  and  agency 
estimates  on  the  front  end  aggravates  cost 
growth.  What  is  needed  is  more  candor  up 
front  in  presenting  programs  to  the  Congress 
and  not  promising  more  than  can  be 
realistically  delivered." 

19.  Statement  of  Jerome  H.  Stolarow,  Director,  Procurement 
and  Systems  Acquisition  Division,  before  the  House 
Committee  on  Government  Operations,  Subcommittee  on 
Legislation  and  National  Security,  on  Weapons  Systems 
Costs,  June  25,  1979. 

"The  question  that  arises  is  why  the  early 
estimates — even  given  the  lack  of  firm  data 
— are  always  so  much  lower  than  the  later 
estimates.  I  think  it  is  fair  to  say  that 
human  nature  plays  a  major  role. 
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"Program  advocates  both  in  DOD  and  industry 
want  to  get  a  program  started.  They  tend  to 
be  highly  optimistic  with  respect  to  costs, 
technical  developmental  problems  and 
operational  characteristics,  i.e.,  the 
proposed  weapon  will  do  wonders  at  a  very 
low  cost.  This  is  expressly  intended  to 
"sell"  both  the  decisionmakers  in  DOD  as 
well  as  the  Congress." 
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AUDIT  WORK  LOCATIONS 

This  review  required  audit  work  at  the  following  locations 

Office  of  the  Secretary  of  Defense — Cost  Analysis 
Improvement  Group,  Washington,  D.  C. 

ARMY 

Apache  Helicopter  Program  Office,  St.  Louis,  Missouri. 

Army  Audit  Agency,  Redstone  Arsenal,  Huntsville,  Alabama. 
Army  Audit  Agency,  Warren,  Michigan. 

Army  Audit  Agency,  Washington,  D.C. 

Bradley  Fighting  Vehicle  System  Program  Office,  Warren, 
Michigan . 

Headquarters,  U.S.  Army,  Washington,  D.C. 

Headquarters,  U.S.  Army  Materiel  Development  and 
Readiness  Command,  Alexandria,  Virginia. 

U.S.  Army  Aviation  Research  and  Development  Command, 

St.  Louis,  Missouri. 

U.S.  Army  Missile  Command,  Redstone  Arsenal,  Huntsville, 
Alabama. 

U.S.  Army  Plant  Representative  Office,  Hughes  Helicopter 
Incorporated,  Culver  City,  California. 

U.S.  Army  Tank  Automotive  Command,  Warren,  Michigan. 

NAVY 

Headquarters,  Naval  Material  Command,  Washington,  D.C. 
LAMPS  MK  III  Program  Office,  Crystal  City,  Virginia. 

LSD-41  Program  Office,  Crystal  City,  Virginia. 

Naval  Air  Systems  Command,  Washington,  D.C. 

Naval  Sea  Systems  Command,  Washington,  D.C. 

Navy  Audit  Service,  Washington,  D.C. 


99 


APPENDIX  VIII 


APPENDIX  VIII 


Office  of  the  Chief  of  Naval  Operations,  Washington,  D.C. 
AIR  FORCE 


Air  Force  Flight  Test  Center,  Edwards  Air  Force  Base, 
California. 

Air  Force  Logistics  Command,  Wright-Patterson  Air  Force 
Base,  Ohio. 

Air  Force  Test  and  Evaluation  Center,  Kirtland  Air  Force 
Base,  New  Mexico. 

Air  Force  Plant  Representative  Office  at  Airborne 
Instruments  Laboratory,  Long  Island,  New  York. 

Air  Force  Plant  Representative  Office  at  Boeing,  Seattle, 
Washington. 

Air  Force  Plant  Representative  Office  at  General  Electric, 
Evendale,  Ohio. 

Air  Force  Plant  Representative  at  Fairchild,  Long  Island, 
New  York. 

Air  Force  Plant  Representative  at  Rockwell,  El  Segundo, 
California . 

Air  Force  Systems  Command,  Andrews  Air  Force  Base, 

Maryland . 

B-IB  System  Program  Office,  Wright-Patterson  Air  Force 
Base,  Ohio. 

Headquarters,  U.S.  Air  Force,  Washington,  D.C. 

T-46A  System  Program  Office,  Wright-Patterson  Air  Force 
Base,  Ohio. 


CONTRACTORS 

Airborne  Instruments  Laboratory,  Long  Island,  New  York. 

Boeing  Military  Airplane  Company,  Seattle,  Washington. 

Fairchild  Republic  Company,  Long  Island,  New  York. 

FMC  Corporation,  San  Jose  Ordnance  Plant,  San  Jose, 
California. 
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Garrett  Turbine  Engine  Company,  Phoenix,  Arizona. 

General  Electric's  Aircraft  Engine  Group,  Lynn, 
Massachusetts . 

General  Electric,  Evendale,  Ohio. 

Hughes  Aircraft  Company,  Electro-Optical  &  Data  Systems 
Group,  Culver  City,  California. 

Hughes  Helicopters  Incorporated,  Culver  City,  California. 

Lockheed  Shipbuilding  and  Construction  Company,  Seattle, 
Washington. 

Rand  Corporation,  Santa  Monica,  California. 

Rockwell  Corporation,  Deluth,  Georgia. 

Rockwell  International,  El  Segundo,  California  and 
Palmdale,  California. 

Sikorsky  Aircraft  Division,  Stratford,  Connecticut. 


OTHERS 


Defense  Contract  Administration  Service  Management 
Agency,  Phoenix,  Arizona. 

Defense  Contract  Audit  Agency  at  Airborne  Instruments 
Laboratory,  Long  Island,  New  York. 

Defense  Contract  Audit  Agency  at  Boeing,  Seattle, 

Washington . 

Defense  Contract  Audit  Agency  at  Fairchild,  Long  Island, 

New  York. 

Defense  Contract  Audit  Agency  at  Garrett,  Phoenix,  Arizona. 

Defense  Contract  Audit  Agency  at  General  Electric, 

Evendale,  Ohio. 

Defense  Contract  Audit  Agency  at  Rockwell,  El  Segundo, 
California. 


Defense  Contract  Audit  Agency,  Los  Angeles,  California. 

/ 

Defense  Contract  Audit  Agency,  St.  Louis,  Missouri. 
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APPEI.’DIX  IX 

AMERICAN  DEFENSE  PREPAREDNESS  ASSOCIATION 

OIDICATSD  TO  PlACi  WTTH  SIC-WTT  THROUCH  DlFUiil  mjAJUONlSS 

xcszvrs  cisTVL  Sim  ?oo.  itoj  north  moori  rrRirr.  arjjnctqn,  v^xc:^aA 

^cvya-ino. 

Novemher  10,  1933 


Mr.  Frank  C.  Conahan 
Director 

National  Security  and  International 
Affairs  Division 

United  States  General  Accounting  Office 
Washington,  D.  C.  20548 

Dear  llr,  Conahan: 

The  American  Defense  Preparedness  Association  appre¬ 
ciates  the  opportunity'  to  review  and  comment  on  a  draft  of 
your  proposed  report  "DoD  Needs  to  Provide  Better  Weapon  Sys¬ 
tems  Cost  Estimates  to  the  Congress”.  We  have  provided  in  the 
subsequent  paragraphs  our  comments  concerning  this  draft 
report. 


APPExnix  i.x 


founded  1919 


Yotir  organization  has  been  on  this  trail  for  over  ten 
(10)  years  and  has  issued  a  variety  of  reports  on  the  subject. 

The  basis  of  yoxir  reports  ir  the  early  years  was  the  SAP.  data. 
This  data  base  has  now  been'  supplemented  by  information  in  the 
Unit  Cost  Reports  required  by  Section  917,  PL  97-86. 

Yoxnr  current  draft  report  recognizes  the  considerable 
on-going  efforts  of  OSD  and  the  Services  to  improve  their  weapon 
systesis  cost  estimates.  An  examination  by  a  layman  of  Appendix 
III  to  your  report  would  convince  him,  I  believe,  that  all  three 
Services  have  developed,  over  time,  intelligently  structured 
systems  for  producing  and  validating  weapon  systems  cost  esti¬ 
mates.  In  Chapter  5  of  the  report  you  also  indicate  that  OSD 
and  the  Services  have  recently  taken  steps  to  improve  the  cost 
estimating  process.  In  light  of  this  effort,,  a  substantial  part 
of  the  report,  based  on  history,  appears  to  be  out  of  date. 

Would  yoxxs  conclusion,  that  cost  estimating  in  the  Defense  Depart 
ment  is  stilly  unacceptable;  have  been  appropriate  if  the  new 
emphasis  had  been  in  existence  at  earlier  times? 


The  ADPA  statement  on  page  3  of  the  Draft  Retort  is 
still  valid. 

Valid  cost  estimating  .depends  on  gathering  all  pertin¬ 
ent  facts  and  proper  analyses  by  competent  people.  Cost  escimac- 
“PS  Is  an ^integrated  activity,  depending  for  success  on  avail- 

several  specialities:  engineers,  production  specialist 
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Mr.  Frank  C.  Conahan 
Page  --2-- 


narketmg  expert price/cost  analysts,  nanagemenc ,  etc.  ^  Hoc 
teaps , ^  called  together  for  individual  actions,  are  not  as 
str active ^as  pemanent  starts  who  bring  to  bear  e^cperience 
o^rnered  froti  related  or  similar  programs'. 

Independent  cost  estimate  teams  are  necessary  at  all 

levels : 

a.  at  the  project -program  manager's  office  to  challenge 
adequacy  of  the  statements  of  the  PM  Staff  and  the 
contractor. 

b.  at  the  command  level  to  challenge  presentation  of 
the  PM. 

c.  at  the  Department  Headqxiarters  level. 

d.  at  OSD  (the  CAIG) . 

Inflation  was  a  principal  cause  of  cost  growth  when 
the  official '  0MB  forecasts  had  to  be  used  for  budgeting  and 
program  management.  The  recent  authority  to  use  more  realistic 
S®®*^**  inflation  ■■mtas  will  diminish  the  problem,  but 
will  not  dispose  of  it  altogether.  Inflation  rates  should 
be  permanently  removed  from  the  political  arena,  as  far  as  de¬ 
fense  is  concerned. 

.  ^  Th®  recent  DoD  actions  for  improvement  of  cost  estimat¬ 

ing  should  help  to  reduce  the  problem.  Defense  testimonv  on  the 
Authorization  Bill  FY  84  (Part  3,  pages  279-379,  House  o’f  Re'ore- 
sentatives)  shows  some  progress. 

,,  ADPA^feels  the  statement  at  the  bottom  of  page  14  that 

purpose  or  documentation  is  to  provide  the  means  for  cracking 
program  cost  estimates  and  enforcing  cost  discipline"  is  mislead^ 
ing.  "nie  need  for  do ctmen cation  is  not,  in  ovir  view,  to  provide 
an  audit  trail  but  rather  to  permit  intelligent  program  management , 
especially  as  managers  change,  so  chat  successive  managers  can 
leam  >7hac  predecessors  have  accomplished,  how  and  why. 

There  is,  in  our  view,  a  need  for  cross-fertilization 
among  the  Services  on  new  ideas  and  better  ways  of  accomolishing 
cost  estimates.  In  this  regard  an  effort  to  ca\ise  and  maintain 
surveillance  on  this  cross-fertilization  by  OSD  we  feel  would  be 
most  benericial. 
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Mr.  Frank  C.  Conahan 
Page  — 3 — 


The  report  indicates  the  Air  Force  and  the  Navy  are 
trying  to  improve"  cost  estimating  capabilities  through 
additional  "recruiting  and  training  of  cost  estimators".  _We 
feel  this  should  be  a  universal  action  by  all  Defense  acci'';itias . 
Cost  estimators  of  one  office  ought  to  be  able  to  function  any¬ 
where  in  DoD. 

We  agree  with  the  nine  (9)  criteria  for  effective 
cost  estimating  referred  to  on  page  2  of  the  report  and  pages 
52  to  54  of  Appendix  I,  namely: 

— broad  participation 
— standardized  structure 
— provide  for  risks 
— recognize  inflation 
— recognize  excluded  cost 
— independently  review  estimate 
— revise  estimate  for  changes 
— identify  tasks 
— valid  data 

Cost  estimates  should  follow  a  uniform  structure  so  chat 
experience  in  orie  case  can.  used  for  later  ■. similar  cases. 
Guidance  should  not  permit  optimistic  assumptions. 

Again,  we  appreciate  -the  opportunity  of  providing  our 
comments  on  your  proposed  report  and  stand  ready  for  further  dis¬ 
cussion  if  you  so  choose. 


Sincerely, 


V 


HenSy  A.  Mile- 
General  ,  U .  S . 
President 


Jr. 

ly  (Retired) 
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ANALYSES  OF  SAR  DATA 


The  following  four  pages  are  the  results  of  analyses  of  the 
December  1982  SARs.  The  36  weapon  systems  in  these  analyses  are 
those  for  which  the  effects  of  the  production  milestone  decision 
(DSARC  III)  could  be  identified  in  the  past  SARs. 

One  set  of  analyses  was  done  on  the  cost  differences 
between  the  system's  development  estimate  (DSARC  II)  and  the  SAR 
current  estimate  associated  with  the  production  decision  (DSARC 
III).  A  second  set  of  analyses  was  done  on  the  cost  differences 
between  the  current  estimate  associated  with  the  production 
decision  (DSARC  III)  and  the  current  estimate  in  the  December 
1982  SAR. 

The  six  cost  variance  categories  used  in  the  analyses  are 
those  that  are  required  to  appear  in  the  SAR  by  DOD  Instruction 
7000.3.  We  did  not  use  the  seventh  required  category — economic 
escalation — because  this  category  involves  0MB  mandated 
inflation  rates  which  DOD  has  no  control  over. 

The  dollar  figures  in  the  analyses  are  derived  by 
determining  the  cost  differences  between  the  two  estimates  for 
each  cost  variance  category,  and  dividing  by  the  total  number  of 
months  between  the  two  estimates  for  all  the  weapons  of  each 
type.  A  similar  procedure  was  used  for  the  percentage 
differences.  The  overall  figures  for  the  36  systems  are  derived 
by  dividing  the  total  change  by  the  total  number  of  months 
between  the  estimates  for  all  36  systems.  Therefore,  the 
figures  in  each  column  do  not  total  to  the  overall  average  for 
the  36  systems. 

In  addition,  because  the  36  weapon  systems  use  different 
base  year  dollars,  the  individual  systems'  costs  are  not  exactly 
comparable  and  the  totals  lack  precision.  However,  this  is  a 
weakness  inherent  in  the  SAR  reporting  system  and  is  true  of  any 
analysis  based  on  the  SARs. 
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OUR  PREVIOUS  RECOMMENDATIONS  ON  SARS 

1.  On  March  2,  1981,  we  issued  a  report  entitled  Recommenda¬ 
tions  to  Improve  Defense  Reporting  on  Weapon  Systems 
(MASAD-81-7 ) ,  in  which  we  recommended  to  the  Congress  that 
it  require  the  Secretary  of  Defense  change  the  SAR  system 
to 


— include  important  systems  that  are  in  advanced 
development; 1 

— expand  the  required  mission  capability  assessment 
statement  to  describe  shortcomings  and  limitations 
of  systems  in  their  expected  operational 
environments; 

— include  planning  estimates  and  a  one-time  variance 
analysis  for  the  planning  and  development  estimates 
in  the  first  report  that  includes  the  development 
estimate; ^ 

— include  a  brief  narrative  section  on  technical  and 
operational  risks; 

— include  cost  estimates  for  categories  of  logistic 
support/additional  procurement  costs  related  to  the 
weapon  system,  such  as  modification  costs,  component 
improvement  costs,  replenishment  spare  costs, 
industrial  facilities/production  base,  simulators, 
consumables,  and  modification  spares;^  and 

— include  a  chart  showing  the  effect  on  the  program 
cost  estimate  of  different  escalation  rates. 

Synopsis  of  our  other  reports  relating  to  SARs  follow: 

2.  "SARs" — Defense  Department  Reports  Should  Provide  More 

Information  to  the  Congress  (PSAD-80-37,  May  9,  1980). 

Important  information  which  would  be  useful  to  the 
Congress  and  to  top-level  DOD  managers  is  being  omitted  from 
DOD's  SARs.  SARs  should  give  better  disclosure  as  to  the 
status  of  acquisition,  should  be  shortened  to  facilitate  use 
by  people  with  little  time,  and  present  data  that  is 
complete,  accurate,  and  not  misleading. 


^We  also  made  these  recommendations  to  the  Secretary  of  Defense 
in  our  March  1975  report  entitled.  How  to  Improve  the  Selected 
Acquisition  Reporting  System  (PSAD-75-63,  Mar.  27,  1975). 
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We  recommended  that  DOD  include  additional  data  in  the 
SARs  such  as  operational  and  technical  risks,  operational 
capability  shortfalls,  and  logistic  support/additional 
procurement  costs  and  explanations  for  changes. 

3.  How  to  Improve  the  Selected  Acquisition  Reporting  System 

(PSAD-75-63,  Mar.  27,  1975) 

Since  its  inception  in  1969,  SAR  has  become  the  key 
recurring  summary  status  report  to  the  Congress  on  the 
progress  of  DOD  weapon  systems  acquisitions.  It  is 
important  that  this  report  provides  full  disclosure  of  the 
status  of  each  system.  We  identified  10  areas  where 
improvements  would  result  in  increased  usefulness  of  SAR  to 
DOD  and  the  Congress.  Some  of  the  areas  identified  were; 

— Performance  characteristics  should  be  related  to 

mission  requirements  and  an  assessment  provided  as  to 
the  extent  the  system  is  expected  to  satisfy  the 
mission  requirements. 

— All  costs  which  are  expected  to  be  expended  for  the 
benefit  of  the  development  and  procurement  of  a 
weapon  system  should  be  included. 

— Considerable  improvement  could  be  made  in  reporting 
logistic  support/additional  procurement  costs.  These 
type  costs  should  be  expanded  to  include  all 
remaining  procurement  costs  related  to  a  program,  but 
not  currently  being  reported  as  program  acquisition 
costs.  These  type  costs  should  be  included  in  the 
cost  section  of  SAR  rather  than  in  a  separate 
section.  In  addition,  the  section  on  logistic 
support/additional  procurement  costs  should  include 
firm  baselines  established  with  footnotes  indicating 
the  basis  for  these  baselines,  and  any  changes  from 
these  baselines  should  be  provided  in  the  form  of  a 
variance  analysis. 

— Significant  pending  decisions  that  may  have  a  major 
effect  on  a  program  should  be  highlighted. 
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4.  Acquisition  of  Major  Weapon  Systems  (B-163058, 

Mar.  18,  1971). 

We  recommended  that  the  Secretary  of  Defense  ensure 
that  each  SAR  (a)  contain  a  summary  statement  regarding  the 
overall  acceptability  of  the  weapon  for  its  mission,  (b) 
recognize  the  relationships  of  other  weapon  systems 
complementary  to  the  subject  systems,  and  (c)  reflect  the 
current  status  of  program  accomplishments. 

5.  Status  of  the  Acquisition  of  Selected  Major  Weapon  Systems 

(B-163058,  Feb.  6,  1970). 

In  April  1969,  the  SAR  system  was  chosen  by  DOD  to  play 
an  important  role  in  monitoring  system  acquisitions  and 
also  as  the  mechanism  for  developing  program  status 
information.  As  with  any  new  reporting  system,  the  SAR 
system  had  serious  shortcomings,  and  several  areas  are  in 
need  of  improvements. 

At  the  time  of  our  examination,  SAR  was  not 
sufficiently  encompassing,  and  therefore  failed  to  disclose 
significant  matters  concerning  the  progress  of  major 
acquisitions.  We  also  noted  inconsistencies  in  the  data 
reported  in  the  SAR.  For  example,  consistency  was  lacking 
in  (1)  the  reporting  of  early  developmental  costs  and  (2) 
treatment  of  costs  attributed  to  inflationary  trends  in  the 
economy. 
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